-- Logs begin at Thu 2022-09-08 12:50:13 UTC, end at Thu 2022-09-08 14:22:11 UTC. -- Sep 08 13:36:26.312734 dv-cinder-847733-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 08 13:36:26.321320 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 08 13:36:26.323603 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 8 13:36:26 2022] *** Sep 08 13:36:26.323603 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 08 13:36:26.323603 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 08 13:36:26.323603 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: nodename: dv-cinder-847733-4-iscsi Sep 08 13:36:26.323603 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: machine: x86_64 Sep 08 13:36:26.323603 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: clock source: unix Sep 08 13:36:26.323603 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: pcre jit disabled Sep 08 13:36:26.323603 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: detected number of CPU cores: 4 Sep 08 13:36:26.323603 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: current working directory: / Sep 08 13:36:26.323603 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: detected binary path: /usr/bin/uwsgi-core Sep 08 13:36:26.323603 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: your processes number limit is 47680 Sep 08 13:36:26.324311 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: your memory page size is 4096 bytes Sep 08 13:36:26.324311 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: detected max file descriptor number: 2048 Sep 08 13:36:26.324311 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: lock engine: pthread robust mutexes Sep 08 13:36:26.324311 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: thunder lock: enabled Sep 08 13:36:26.324311 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 08 13:36:26.324311 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 08 13:36:26.363247 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: Python main interpreter initialized at 0x55cf5f1949c0 Sep 08 13:36:26.363247 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: python threads support enabled Sep 08 13:36:26.363247 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: your server socket listen backlog is limited to 100 connections Sep 08 13:36:26.363247 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: your mercy for graceful operations on workers is 90 seconds Sep 08 13:36:26.363598 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: mapped 403077 bytes (393 KB) for 2 cores Sep 08 13:36:26.363673 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: *** Operational MODE: preforking *** Sep 08 13:36:26.363924 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: *** uWSGI is running in multiple interpreter mode *** Sep 08 13:36:26.363924 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: spawned uWSGI master process (pid: 93161) Sep 08 13:36:26.364120 dv-cinder-847733-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 08 13:36:26.367787 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: spawned uWSGI worker 1 (pid: 93172, cores: 1) Sep 08 13:36:26.367787 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: spawned uWSGI worker 2 (pid: 93173, cores: 1) Sep 08 13:36:26.367787 dv-cinder-847733-4-iscsi devstack@keystone.service[93161]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 08 13:36:30.693690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:36:30.693690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: from cryptography.utils import int_from_bytes Sep 08 13:36:30.694474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:36:30.694474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: from cryptography.utils import int_from_bytes Sep 08 13:36:30.694586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:36:30.694586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: from cryptography.utils import int_from_bytes Sep 08 13:36:30.695033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:36:30.695033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: from cryptography.utils import int_from_bytes Sep 08 13:36:30.754419 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 13:36:30.754584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 13:36:30.754668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 13:36:30.754761 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 13:36:30.754837 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] command line args: [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 13:36:30.754949 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] command line args: [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 13:36:30.755027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 13:36:30.755105 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 13:36:30.755183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ================================================================================ {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 13:36:30.755256 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ================================================================================ {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 13:36:30.755330 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] admin_token = **** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.755408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] admin_token = **** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.755480 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] config_dir = [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.755551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] config_dir = [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.755620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.755692 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.755769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] config_source = [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.755851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] config_source = [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.755928 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] debug = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.756006 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] debug = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.756084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.756542 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.756542 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.756706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.756774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.756835 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.756901 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.756969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.756969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.757079 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.757142 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] list_limit = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.757206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] list_limit = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.757269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] log_config_append = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.757337 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] log_config_append = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.757396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.757458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.757520 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] log_dir = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.757583 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] log_dir = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.757640 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] log_file = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.757707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] log_file = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.757765 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.757822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.757878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.757940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.757997 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.758064 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.758123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.758182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.758241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.758299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.758364 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.758429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.758493 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.758556 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.758620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.758702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.758762 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.758824 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.758887 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.758998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.758998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.759117 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.759176 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.759239 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.759306 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.759377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.759441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.759507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.759572 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.759631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.759688 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.91/identity {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.759746 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.91/identity {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.759813 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] publish_errors = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.759872 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] publish_errors = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.759927 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.760038 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.760099 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.760156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.760218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.760281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.760361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.760440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.760506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.760565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.760621 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] standard_threads = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.760687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] standard_threads = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.761109 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.761175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.761232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.761295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.761354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.761416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.761532 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] use_journal = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.761630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] use_journal = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.761740 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] use_json = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.761818 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] use_json = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.761889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] use_stderr = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.761889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] use_syslog = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.762020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] use_stderr = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.762097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.762167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] use_syslog = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.762167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:30.762297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.762297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.762508 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.762508 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.762642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.762719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.762789 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.762866 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.762954 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.763025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.763093 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.763172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.763249 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.763330 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.763407 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.763484 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.763554 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.763622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.763690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.763770 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.763842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.763994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.764081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.764232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.764318 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.764485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.764577 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.764727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.764811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.764980 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.765058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.765230 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.765308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.765469 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.765546 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.765729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.765835 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.765964 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.766076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.766184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.766303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.766444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.766522 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.766706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.766784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.766988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.767112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.767241 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.767322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.767473 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.767571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.767694 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.767808 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.767926 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.768047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.768163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.768284 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.768412 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.768525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.768662 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.768778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.768893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.769032 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.769110 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.769269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.769352 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.769523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.769609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.769758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.769833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.770016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.770110 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.770256 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.770325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.770503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.770633 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.770803 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.770984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.771255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.771338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.771457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.771570 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.771684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.771833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.771960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.772075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.772196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.772319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.772424 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.772536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.772660 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.772764 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.772876 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.772989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.773100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.773212 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.773333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.773459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.773574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.773694 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.773809 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.773922 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.774034 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.774155 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.774270 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.774384 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.774496 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.774608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.774721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.774846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.774974 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.775088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.775201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.775324 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.775440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.775562 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.775677 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.775790 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.775905 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.776084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.776182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.776286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.776384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.776384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.776573 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.776677 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.776774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.776879 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.776975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.777071 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.777174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.777271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.777372 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.777475 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.777581 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.777685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.777781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.777876 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.777971 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.778071 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.778185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.778284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.778379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.778474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.778569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.778716 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.778925 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.779031 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.779128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.779223 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.779318 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.779444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.783609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.783770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.783908 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.784035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.784162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.784288 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.784436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.784581 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.784705 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.784829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.784953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.785077 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.785212 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.785359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.785495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.785586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.785747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.785862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.786011 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.786160 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.786276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.786353 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.786426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.786497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.786569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.786651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.786726 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.786804 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.786877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.786977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.787050 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.787134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.787208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.787279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.787349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.787419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.787489 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.787570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.787643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.787714 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.787784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.787854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.787924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.788003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.788113 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.788185 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.788246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.788368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.788444 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.788525 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.788597 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.788668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.788747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.788819 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.788895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.788978 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.789052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.789122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.789193 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.789262 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] auth.external = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.789340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.789448 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.789529 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.789610 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.789685 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.789757 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.789828 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] auth.external = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.789898 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] auth.password = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.789968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.790046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] auth.token = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.790119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.790198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.790198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.790327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.790404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.790477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] auth.password = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.790548 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.790693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] auth.token = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.790776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.790850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.790964 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.791042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.791114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.791192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.791266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] credential.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.791343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.791427 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.791501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.791573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.791645 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.791722 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.791800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.791881 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.791954 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] credential.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.792026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.792097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.792167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.792244 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.792325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.792398 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.792469 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.792541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.792617 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.793185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.793274 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.793348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.793420 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.793491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.793562 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.793633 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.793787 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.793863 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.793935 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.794012 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.794085 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.794156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.794234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] federation.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.794322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.794406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.794480 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.794695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.794777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.794922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.795012 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.795087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.795159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] federation.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.795236 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.795316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.795398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.795471 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.795543 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.795615 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.795711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.795791 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.795968 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.796048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.796207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.796289 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.796456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.796535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.796693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.796773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.796943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.797022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.797190 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.797269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.797428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.797509 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.797665 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.797744 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.797912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.797998 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.798143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.798262 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.798381 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.798460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.798623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.798708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.798875 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.799023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.799166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.799390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.799390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.799610 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.799729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.799844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.799958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.800072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.800187 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.800313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.800429 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.800543 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.800658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.800772 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.800887 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.801053 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.801167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.801263 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.801411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.801523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.801650 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.801764 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.801864 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.802013 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.802119 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.802293 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.802399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.802543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.802654 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.802753 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.802849 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.803017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.803124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.809072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.809186 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.809268 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.809341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.809341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.809487 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.809567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.809641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.809712 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.809783 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.809853 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.809929 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.810010 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.810084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.810154 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.810225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.810296 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.810367 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.810446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.810519 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.810590 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.810661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.810732 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.810851 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.810993 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.811071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.811143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.811214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.811214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.811341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.811421 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.811494 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.811565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.811636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.811707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.811777 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.811857 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.811929 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.811929 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.812056 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.812126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.812197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.812276 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.812361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.812361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.812492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.812562 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.812633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.812713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.812786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.812857 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.812928 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.812999 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.813069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.813148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.813221 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.813292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.813362 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.813433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.813502 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.813581 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.813654 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.813724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.813795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.813865 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.813946 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.814020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.814091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.814091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.814226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.814296 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.814374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.814452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.814525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.814596 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.814666 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.814736 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.814814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.814887 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.814992 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.815062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.815133 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.815133 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.815271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.815271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.815401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.815471 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.815541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.815611 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.815689 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.815762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.815833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.815904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.815975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.816045 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.816122 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.816195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.816265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.816336 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.816407 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.816477 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.816555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.816656 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.816723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.816781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.816900 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.816976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.817126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.817202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.817273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.817344 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.817415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.817485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.817563 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.817636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.817707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.817777 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.817847 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.817917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.817996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.818068 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.818225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] resource.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.818306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.818467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.818547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.818717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.818799 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.818978 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.819062 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.819218 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.819296 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.819466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.819548 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.819700 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.819778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] resource.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.819949 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.820060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.820190 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.820271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.820432 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.820518 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.820668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] role.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.820747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.820920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] role.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.821009 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.821152 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.821231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.821402 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.821482 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.821652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.821733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.821899 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.821979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.822140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] role.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.822217 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.822386 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.822466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] role.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.822622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.822700 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.823146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.823232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.823402 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.823483 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.823638 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.823716 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.823887 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.824460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.824661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.824742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.824912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.825103 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.825182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.825462 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.825576 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.825814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.825892 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.826065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.826190 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.826387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.826579 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.826701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.826817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.826953 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.827071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.827196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.827313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.827428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.827604 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.827722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.827838 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.827962 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.828078 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.828192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.828307 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.828446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.828565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.828688 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.828805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.828942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.829062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.829177 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.829291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.829414 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.829530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.829668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.829785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.829900 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.830015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.830146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] token.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.830263 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.830378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.830491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.830605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.830719 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.830841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.831008 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.831124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.831238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.831352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.831479 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.831596 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.831711 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.831847 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.831967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.832083 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.832210 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.832326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.832466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] token.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.832565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.832661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.832756 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.832862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.832992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.833098 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.833239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.833344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.833520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.833633 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.833752 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.833867 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.833994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.834101 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.834257 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.834361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.834514 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.834627 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.connection = **** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.834755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.834859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.835024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.835129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.835261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.837241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.837350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.837465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.837562 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.837676 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.837790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.837913 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.838042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.838168 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.838284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.838401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.838517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.838632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.connection = **** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.838824 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.839009 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.839129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.839285 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.839406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.839521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.840164 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.840287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.840402 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.840518 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.840633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.840747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.840870 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.840986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.841101 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.841215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.841330 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.841330 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 13:36:30.841555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.841555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.841555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.841555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.841555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.841555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.841555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.841555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.841555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.842479 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.842479 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 13:36:30.868318 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.868544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.868769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.870254 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.870492 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.870717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.877111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.879287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.906178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.906403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.911250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.911490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.967908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.968085 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.024277 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.024431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.024586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.024755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.188346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.188578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.188808 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.189133 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.189250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.189330 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.189406 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.189481 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.189554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.189638 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.189743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.189964 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.215956 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.216083 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.216237 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.216320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.216394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.216467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.216615 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.216736 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.216819 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.216993 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.217074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.217224 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.217306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.217452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.228741 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.229347 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.242518 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.242692 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.242803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.242963 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.243050 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.243200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.243283 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.243454 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.243538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.243673 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.243752 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.243894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.243972 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.244139 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:31.266720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.266957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 13:36:31.267044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.267263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 13:36:31.267358 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.267494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 13:36:31.267608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.267769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.267849 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 13:36:31.268033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.268122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.268294 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.268375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.268562 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.268642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.268829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.278978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.279221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.279494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.279619 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.279782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.279906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.280184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.280468 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.286867 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.287695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.289856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.290145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.290435 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.290678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.290757 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.291025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.291109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.291291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.291372 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.291565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.291674 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.291952 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.292032 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.292223 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.292304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.292502 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.292588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.293326 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.293451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.293757 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.294062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.294505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.319345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.319499 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.320832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.320917 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.321123 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.321230 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.326735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.326857 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.327045 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.327176 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.327334 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.327468 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.333720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.333862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.334010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.334153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.339856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.340044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.340157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.340347 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.340456 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.340650 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.340754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.340973 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.341166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.341247 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.341457 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.341538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.341740 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.341820 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.342021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.342110 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.357384 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.357664 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.358491 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.358780 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.361581 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.362772 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.364777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.365082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.366101 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.366419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.369253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.370696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.372937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.374545 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.376154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.376461 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.376752 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.377043 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.377344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.377630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.377858 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.378166 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.378544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.378858 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.379187 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.379494 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.393829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.394121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.394391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.398143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.398234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.398234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.398234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.398234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.401598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.403251 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.403784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.404772 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.405072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.405823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.406968 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.407273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.408062 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.408360 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.408652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.408939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.409326 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.410288 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.410569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.410859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.411183 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.411585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.420427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.420715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.421008 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.421296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.426785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.426785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.426785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.426999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.426999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.426999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.426999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.426999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.426999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.427320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.439114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.441389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.442014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.444345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.444908 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.446488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.447319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.448243 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.448527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.448924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.449100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.451111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.451401 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.451805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.455792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.457263 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.463753 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.463753 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.463948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.463948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.463948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.463948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.464200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.464200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.464200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.464200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.469038 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.469335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.469625 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.471682 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.471977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.472337 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.476731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.477024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.478214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.478214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.478214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.478214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.478483 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.478924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.479011 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.479784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.480875 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:31.480946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.480946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.481056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.481116 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.481187 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.481251 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.481311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.481311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.481419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.481732 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.482029 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.482305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.482595 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.482869 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.483157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.483429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.483783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:31.515818 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55cf5f1949c0 pid: 93172 (default app) Sep 08 13:36:31.519411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55cf5f1949c0 pid: 93173 (default app) Sep 08 13:36:31.519514 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534f67c-eb69-4947-9bcc-f1531da92151 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:31.519774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534f67c-eb69-4947-9bcc-f1531da92151 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:31.520053 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534f67c-eb69-4947-9bcc-f1531da92151 None None] PATH_INFO: `/v3` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:31.524268 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1/1] 10.222.0.91 () {50 vars in 791 bytes} [Thu Sep 8 13:36:31 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:40.064362 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36476ed-7001-41a5-abec-664ec6500c67 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:40.064900 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36476ed-7001-41a5-abec-664ec6500c67 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:40.064900 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36476ed-7001-41a5-abec-664ec6500c67 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:40.069325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1/2] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:36:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:40.074873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311f540-e6a0-4c46-9275-036bb20ce907 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:40.075340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311f540-e6a0-4c46-9275-036bb20ce907 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:40.075435 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311f540-e6a0-4c46-9275-036bb20ce907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:40.144911 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: INFO dbcounter [None req-0311f540-e6a0-4c46-9275-036bb20ce907 None None] Registered counter for database keystone Sep 08 13:36:40.165879 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG oslo_db.sqlalchemy.engines [None req-0311f540-e6a0-4c46-9275-036bb20ce907 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93172) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:36:40.172525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writer thread running {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:36:40.351745 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG passlib.handlers.bcrypt [None req-0311f540-e6a0-4c46-9275-036bb20ce907 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93172) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 08 13:36:40.352074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG passlib.handlers.bcrypt [None req-0311f540-e6a0-4c46-9275-036bb20ce907 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93172) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 08 13:36:40.410499 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-0311f540-e6a0-4c46-9275-036bb20ce907 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:40.412112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0311f540-e6a0-4c46-9275-036bb20ce907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:40.540318 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 2/3] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 08 13:36:41.052035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36476ed-7001-41a5-abec-664ec6500c67 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:41.124486 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: INFO dbcounter [None req-a36476ed-7001-41a5-abec-664ec6500c67 None None] Registered counter for database keystone Sep 08 13:36:41.144279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG oslo_db.sqlalchemy.engines [None req-a36476ed-7001-41a5-abec-664ec6500c67 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93173) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:36:41.150579 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writer thread running {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:36:41.270266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36476ed-7001-41a5-abec-664ec6500c67 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:41.274672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7cfb7f-a2b4-4140-b396-77043279adc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:41.275332 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7cfb7f-a2b4-4140-b396-77043279adc8 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:41.275606 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7cfb7f-a2b4-4140-b396-77043279adc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:41.275904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7cfb7f-a2b4-4140-b396-77043279adc8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:41.284302 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7cfb7f-a2b4-4140-b396-77043279adc8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:41.337922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9b7cfb7f-a2b4-4140-b396-77043279adc8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 08 13:36:41.338199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7cfb7f-a2b4-4140-b396-77043279adc8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:41.348132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 2/4] 10.222.0.91 () {58 vars in 1324 bytes} [Thu Sep 8 13:36:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 13:36:41.354093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311f540-e6a0-4c46-9275-036bb20ce907 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:41.371236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311f540-e6a0-4c46-9275-036bb20ce907 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:41.376208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf49c568-b8f1-40a0-b731-73ed535f8082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:41.377303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49c568-b8f1-40a0-b731-73ed535f8082 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:41.377584 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49c568-b8f1-40a0-b731-73ed535f8082 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:41.377856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49c568-b8f1-40a0-b731-73ed535f8082 admin admin] PATH_INFO: `/v3/projects/189571d729d54bff900838adb0030c5b` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:41.379516 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf49c568-b8f1-40a0-b731-73ed535f8082 admin admin] RBAC: Authorizing `identity:get_project(project_id=189571d729d54bff900838adb0030c5b)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:41.431153 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bf49c568-b8f1-40a0-b731-73ed535f8082 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 08 13:36:41.431498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf49c568-b8f1-40a0-b731-73ed535f8082 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:41.436643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 3/5] 10.222.0.91 () {56 vars in 1207 bytes} [Thu Sep 8 13:36:41 2022] GET /identity/v3/projects/189571d729d54bff900838adb0030c5b => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:43.543166 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaace90-7971-49ee-a017-f6fc67b9514a None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:43.544272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaace90-7971-49ee-a017-f6fc67b9514a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:43.544272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaace90-7971-49ee-a017-f6fc67b9514a None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:43.548287 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 3/6] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:36:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:43.553749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f32ab-ac2f-47a0-aef3-63bf0a050f81 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:43.554167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f32ab-ac2f-47a0-aef3-63bf0a050f81 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:43.554301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f32ab-ac2f-47a0-aef3-63bf0a050f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:43.574902 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a14f32ab-ac2f-47a0-aef3-63bf0a050f81 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:43.576439 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a14f32ab-ac2f-47a0-aef3-63bf0a050f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:43.584175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 4/7] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 08 13:36:43.856095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaace90-7971-49ee-a017-f6fc67b9514a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:43.868308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaace90-7971-49ee-a017-f6fc67b9514a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:43.873115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a1d1b-8ffc-4c20-9009-8856722d05d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:43.873709 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a1d1b-8ffc-4c20-9009-8856722d05d8 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:43.873986 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a1d1b-8ffc-4c20-9009-8856722d05d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:43.874257 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a1d1b-8ffc-4c20-9009-8856722d05d8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:43.881826 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a1d1b-8ffc-4c20-9009-8856722d05d8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:43.883624 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0a1d1b-8ffc-4c20-9009-8856722d05d8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:43.892977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 4/8] 10.222.0.91 () {58 vars in 1324 bytes} [Thu Sep 8 13:36:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:36:43.898386 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f32ab-ac2f-47a0-aef3-63bf0a050f81 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:43.907420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f32ab-ac2f-47a0-aef3-63bf0a050f81 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:43.911734 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275e364-0ef6-4500-ba2a-6fb178a128ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:43.912349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275e364-0ef6-4500-ba2a-6fb178a128ea admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:43.912626 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275e364-0ef6-4500-ba2a-6fb178a128ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:43.912897 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275e364-0ef6-4500-ba2a-6fb178a128ea admin admin] PATH_INFO: `/v3/users/c14eb0bf645e49f887a8d288d37d1491` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:43.914178 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275e364-0ef6-4500-ba2a-6fb178a128ea admin admin] RBAC: Authorizing `identity:get_user(user_id=c14eb0bf645e49f887a8d288d37d1491)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:43.916042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275e364-0ef6-4500-ba2a-6fb178a128ea admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:43.917551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 5/9] 10.222.0.91 () {56 vars in 1198 bytes} [Thu Sep 8 13:36:43 2022] GET /identity/v3/users/c14eb0bf645e49f887a8d288d37d1491 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:46.134135 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3bd10-bea0-44c4-8ad4-57d6fbb64d5c None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:46.134680 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3bd10-bea0-44c4-8ad4-57d6fbb64d5c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:46.134680 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3bd10-bea0-44c4-8ad4-57d6fbb64d5c None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:46.136038 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 5/10] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:46.141198 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61765f5-368a-4c96-a796-629e40a31491 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:46.141619 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61765f5-368a-4c96-a796-629e40a31491 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:46.141760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61765f5-368a-4c96-a796-629e40a31491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:46.160412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f61765f5-368a-4c96-a796-629e40a31491 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:46.161812 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f61765f5-368a-4c96-a796-629e40a31491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:46.178784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e888c6-2ab0-4a1f-8a92-a1ac168960ad None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:46.179113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e888c6-2ab0-4a1f-8a92-a1ac168960ad None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:46.179392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e888c6-2ab0-4a1f-8a92-a1ac168960ad None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:46.180709 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 6/11] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:36:46.185773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eef8e2-d912-447e-a2b2-e480caeaa467 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:46.186059 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eef8e2-d912-447e-a2b2-e480caeaa467 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:46.186339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eef8e2-d912-447e-a2b2-e480caeaa467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:46.228418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG passlib.handlers.bcrypt [None req-32eef8e2-d912-447e-a2b2-e480caeaa467 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93173) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 08 13:36:46.228820 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG passlib.handlers.bcrypt [None req-32eef8e2-d912-447e-a2b2-e480caeaa467 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93173) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 08 13:36:46.244715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 6/12] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:36:46.278357 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-32eef8e2-d912-447e-a2b2-e480caeaa467 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:46.280030 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-32eef8e2-d912-447e-a2b2-e480caeaa467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:46.381710 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 7/13] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:36:46.511406 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61765f5-368a-4c96-a796-629e40a31491 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:46.537772 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:46.615649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61765f5-368a-4c96-a796-629e40a31491 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:46.648528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dc1d4-7017-445f-bf13-57d2552b6d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:46.649153 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dc1d4-7017-445f-bf13-57d2552b6d03 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:46.649431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dc1d4-7017-445f-bf13-57d2552b6d03 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:46.649702 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dc1d4-7017-445f-bf13-57d2552b6d03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:46.655311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eef8e2-d912-447e-a2b2-e480caeaa467 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:46.661825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4dc1d4-7017-445f-bf13-57d2552b6d03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:36:46.670714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-3b4dc1d4-7017-445f-bf13-57d2552b6d03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:36:46.670714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:36:46.670714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:36:46.670714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:36:46.670714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:36:46.670714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:36:46.670714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:36:46.670714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:36:46.670714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:36:46.670714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:36:46.670714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:36:46.670714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:36:46.670714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:36:46.670714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:36:46.670714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:36:46.670714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:36:46.671838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:36:46.671838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:36:46.671838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:36:46.671838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:36:46.671838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:36:46.671838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:36:46.671838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:36:46.671838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:36:46.671838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:36:46.671838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:36:46.671838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:36:46.671838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:36:46.671838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:36:46.671838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:36:46.671838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:36:46.671838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:36:46.672831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:36:46.672831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:36:46.672831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:36:46.672831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:36:46.672831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:36:46.672831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:36:46.672831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:36:46.672831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:36:46.672831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:36:46.672831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:36:46.672831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:36:46.672831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:36:46.672831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 7/14] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:36:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:36:46.677876 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dc1d4-7017-445f-bf13-57d2552b6d03 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:46.757848 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eef8e2-d912-447e-a2b2-e480caeaa467 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:46.758751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:46.760017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:46.777659 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dc1d4-7017-445f-bf13-57d2552b6d03 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:46.794239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5a8cf-6fd9-4cbd-b248-d8d9b3313851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:46.803575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5a8cf-6fd9-4cbd-b248-d8d9b3313851 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:46.803575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5a8cf-6fd9-4cbd-b248-d8d9b3313851 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:46.803575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5a8cf-6fd9-4cbd-b248-d8d9b3313851 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:46.811206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad8f77-f66f-4a4d-98be-10b574a20fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:46.811806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad8f77-f66f-4a4d-98be-10b574a20fba None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:46.812083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad8f77-f66f-4a4d-98be-10b574a20fba None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:46.812363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad8f77-f66f-4a4d-98be-10b574a20fba None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:46.813025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad8f77-f66f-4a4d-98be-10b574a20fba None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:36:46.813324 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad8f77-f66f-4a4d-98be-10b574a20fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:46.814721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad8f77-f66f-4a4d-98be-10b574a20fba None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:46.827287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 8/15] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:36:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:36:46.833225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad8f77-f66f-4a4d-98be-10b574a20fba None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:46.862852 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5a8cf-6fd9-4cbd-b248-d8d9b3313851 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:46.864417 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5a8cf-6fd9-4cbd-b248-d8d9b3313851 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:46.935275 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad8f77-f66f-4a4d-98be-10b574a20fba None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:46.966628 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 8/16] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:36:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:36:46.968754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30776ba0-bd86-4c83-97c5-45fc7d78d918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:46.969381 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30776ba0-bd86-4c83-97c5-45fc7d78d918 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:46.969660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30776ba0-bd86-4c83-97c5-45fc7d78d918 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:46.969932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30776ba0-bd86-4c83-97c5-45fc7d78d918 None admin] PATH_INFO: `/v3/users/c14eb0bf645e49f887a8d288d37d1491` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:46.971081 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30776ba0-bd86-4c83-97c5-45fc7d78d918 None admin] RBAC: Authorizing `identity:get_user(user_id=c14eb0bf645e49f887a8d288d37d1491)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:46.972170 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5a8cf-6fd9-4cbd-b248-d8d9b3313851 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:46.972534 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30776ba0-bd86-4c83-97c5-45fc7d78d918 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:46.974160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 9/17] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:36:46 2022] GET /identity/v3/users/c14eb0bf645e49f887a8d288d37d1491 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:46.975234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:46.979349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30776ba0-bd86-4c83-97c5-45fc7d78d918 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:46.995309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:47.070699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5a8cf-6fd9-4cbd-b248-d8d9b3313851 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:47.081228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30776ba0-bd86-4c83-97c5-45fc7d78d918 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:47.104318 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01052026-ff31-4ac6-90c5-da10acee5ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:47.105156 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01052026-ff31-4ac6-90c5-da10acee5ee9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:47.105436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01052026-ff31-4ac6-90c5-da10acee5ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:47.105710 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01052026-ff31-4ac6-90c5-da10acee5ee9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:47.115460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ac305-7e7c-448a-9202-2877ebd4033c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:47.116091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ac305-7e7c-448a-9202-2877ebd4033c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:47.116370 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ac305-7e7c-448a-9202-2877ebd4033c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:47.116642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ac305-7e7c-448a-9202-2877ebd4033c None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:47.117755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ac305-7e7c-448a-9202-2877ebd4033c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:47.119247 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ac305-7e7c-448a-9202-2877ebd4033c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:47.120810 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 10/18] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:36:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:36:47.124685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01052026-ff31-4ac6-90c5-da10acee5ee9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:47.126149 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01052026-ff31-4ac6-90c5-da10acee5ee9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:47.127173 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ac305-7e7c-448a-9202-2877ebd4033c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:47.145483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-01052026-ff31-4ac6-90c5-da10acee5ee9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:36:47.145483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:36:47.145483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:36:47.145483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:36:47.145483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:36:47.145483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:36:47.145483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:36:47.145483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:36:47.145483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:36:47.145483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:36:47.145483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:36:47.145483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:36:47.145483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:36:47.145483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:36:47.145483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:36:47.145483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:36:47.147005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:36:47.148151 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:36:47.148151 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:36:47.148151 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:36:47.148151 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 9/19] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:36:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:36:47.152394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01052026-ff31-4ac6-90c5-da10acee5ee9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:47.209980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:47.213867 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:47.233542 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ac305-7e7c-448a-9202-2877ebd4033c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:47.254971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01052026-ff31-4ac6-90c5-da10acee5ee9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:47.267322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7dc70-1176-43b8-b97e-a45e02c7218a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:47.268020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7dc70-1176-43b8-b97e-a45e02c7218a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:47.268303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7dc70-1176-43b8-b97e-a45e02c7218a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:47.268579 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7dc70-1176-43b8-b97e-a45e02c7218a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:47.269381 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7dc70-1176-43b8-b97e-a45e02c7218a None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.domain.id=default, user.id=c14eb0bf645e49f887a8d288d37d1491) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:36:47.269760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7dc70-1176-43b8-b97e-a45e02c7218a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:47.271169 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7dc70-1176-43b8-b97e-a45e02c7218a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:47.279006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 11/20] 10.222.0.91 () {56 vars in 1322 bytes} [Thu Sep 8 13:36:47 2022] GET /identity/v3/role_assignments?user.id=c14eb0bf645e49f887a8d288d37d1491&scope.domain.id=default&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:36:47.289624 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4af980-b8c8-499f-8dcf-891a9413e2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:47.290251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4af980-b8c8-499f-8dcf-891a9413e2e0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:47.290536 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4af980-b8c8-499f-8dcf-891a9413e2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:47.290812 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4af980-b8c8-499f-8dcf-891a9413e2e0 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:47.291511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4af980-b8c8-499f-8dcf-891a9413e2e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:36:47.291844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4af980-b8c8-499f-8dcf-891a9413e2e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:47.293295 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4af980-b8c8-499f-8dcf-891a9413e2e0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:47.314130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 10/21] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:36:49.736756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96018c-2ec1-4ab6-901c-be00d344eb89 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:49.737232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96018c-2ec1-4ab6-901c-be00d344eb89 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:49.737232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96018c-2ec1-4ab6-901c-be00d344eb89 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:49.739051 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 12/22] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:49.744360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6a620-f7d4-47dc-846f-e639508dff81 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:49.744822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6a620-f7d4-47dc-846f-e639508dff81 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:49.744898 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6a620-f7d4-47dc-846f-e639508dff81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:49.775958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-60d6a620-f7d4-47dc-846f-e639508dff81 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:49.777194 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-60d6a620-f7d4-47dc-846f-e639508dff81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:49.780041 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96210835-c604-4a80-9db8-2d62e670b57c None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:49.780326 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96210835-c604-4a80-9db8-2d62e670b57c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:49.780600 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96210835-c604-4a80-9db8-2d62e670b57c None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:49.782242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 13/23] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:49.787469 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267a7d0-2be1-4c35-bc4a-c73333658c0d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:49.787754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267a7d0-2be1-4c35-bc4a-c73333658c0d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:49.788036 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267a7d0-2be1-4c35-bc4a-c73333658c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:49.819342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e267a7d0-2be1-4c35-bc4a-c73333658c0d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:49.820559 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e267a7d0-2be1-4c35-bc4a-c73333658c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:49.906860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 11/24] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:36:49.928455 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 14/25] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:36:50.203315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6a620-f7d4-47dc-846f-e639508dff81 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:50.261973 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:50.335640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6a620-f7d4-47dc-846f-e639508dff81 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:50.389318 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d631cb-28be-4f43-ab03-3b2333e61b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:50.389981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d631cb-28be-4f43-ab03-3b2333e61b6a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:50.390262 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d631cb-28be-4f43-ab03-3b2333e61b6a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:50.390539 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d631cb-28be-4f43-ab03-3b2333e61b6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:50.406921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d631cb-28be-4f43-ab03-3b2333e61b6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:50.408430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d631cb-28be-4f43-ab03-3b2333e61b6a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:50.424195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-a5d631cb-28be-4f43-ab03-3b2333e61b6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:36:50.424195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:36:50.424195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:36:50.424195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:36:50.424195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:36:50.424195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:36:50.424195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:36:50.424195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:36:50.424195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:36:50.424195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:36:50.424195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:36:50.424195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:36:50.424195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:36:50.424195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:36:50.424195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:36:50.424195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:36:50.425084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:36:50.425880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:36:50.425880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:36:50.425880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:36:50.425880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 12/26] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:36:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:36:50.431409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267a7d0-2be1-4c35-bc4a-c73333658c0d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:50.525487 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a5072-7879-4925-9b14-e57d6f15d8c8 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:50.525822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a5072-7879-4925-9b14-e57d6f15d8c8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:50.526051 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a5072-7879-4925-9b14-e57d6f15d8c8 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:50.527575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 13/27] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:36:50.535775 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a5072-7879-4925-9b14-e57d6f15d8c8 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:50.557102 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:50.588229 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267a7d0-2be1-4c35-bc4a-c73333658c0d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:50.630625 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecc61b-64e9-41e7-b603-8f8a36487f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:50.631732 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecc61b-64e9-41e7-b603-8f8a36487f31 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:50.632160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecc61b-64e9-41e7-b603-8f8a36487f31 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:50.632822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecc61b-64e9-41e7-b603-8f8a36487f31 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:50.634076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ecc61b-64e9-41e7-b603-8f8a36487f31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:36:50.634541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ecc61b-64e9-41e7-b603-8f8a36487f31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:50.636138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ecc61b-64e9-41e7-b603-8f8a36487f31 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:50.661088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 15/28] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:36:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:50.662684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8d777-7d64-43aa-982e-6ef88ec46303 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:50.663307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8d777-7d64-43aa-982e-6ef88ec46303 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:50.663714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8d777-7d64-43aa-982e-6ef88ec46303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:50.688578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-eeb8d777-7d64-43aa-982e-6ef88ec46303 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:50.689919 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:50.691733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-eeb8d777-7d64-43aa-982e-6ef88ec46303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:50.695157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a5072-7879-4925-9b14-e57d6f15d8c8 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:50.733523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a471ed-1e14-49d6-8f59-7abe4bf2d5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:50.734765 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a471ed-1e14-49d6-8f59-7abe4bf2d5dc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:50.735256 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a471ed-1e14-49d6-8f59-7abe4bf2d5dc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:50.735767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a471ed-1e14-49d6-8f59-7abe4bf2d5dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:50.752772 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a471ed-1e14-49d6-8f59-7abe4bf2d5dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:36:50.761609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-b2a471ed-1e14-49d6-8f59-7abe4bf2d5dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:36:50.761609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:36:50.761609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:36:50.761609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:36:50.761609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:36:50.761609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:36:50.761609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:36:50.761609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:36:50.761609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:36:50.761609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:36:50.761609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:36:50.761609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:36:50.761609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:36:50.761609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:36:50.761609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:36:50.761609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:36:50.763081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:36:50.763081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:36:50.763081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:36:50.763081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:36:50.763081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:36:50.763081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:36:50.763081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:36:50.763081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:36:50.763081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:36:50.763081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:36:50.763081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:36:50.763081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:36:50.763081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:36:50.763081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:36:50.763081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:36:50.763081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:36:50.763850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:36:50.763850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:36:50.763850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:36:50.763850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:36:50.763850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:36:50.763850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:36:50.763850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:36:50.763850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:36:50.763850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:36:50.763850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:36:50.763850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:36:50.763850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:36:50.765218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 14/29] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:36:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:36:50.766697 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4c317-38c5-45c6-af57-e311b95703fd None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:50.767271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4c317-38c5-45c6-af57-e311b95703fd None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:50.767763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4c317-38c5-45c6-af57-e311b95703fd None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:50.769565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 15/30] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:36:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:50.770638 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4c317-38c5-45c6-af57-e311b95703fd None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:50.791036 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 16/31] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:36:50.792629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8d777-7d64-43aa-982e-6ef88ec46303 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:50.916276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:50.955593 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4c317-38c5-45c6-af57-e311b95703fd None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:50.976392 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8d777-7d64-43aa-982e-6ef88ec46303 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:51.017446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69570550-524f-4a42-b162-4dcee369333c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:51.017976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69570550-524f-4a42-b162-4dcee369333c None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.018260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69570550-524f-4a42-b162-4dcee369333c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.018531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69570550-524f-4a42-b162-4dcee369333c None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.020727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69570550-524f-4a42-b162-4dcee369333c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:51.022205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69570550-524f-4a42-b162-4dcee369333c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:51.042324 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ed3dc-9bfb-4db0-939e-3ef59eebf3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:51.042971 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ed3dc-9bfb-4db0-939e-3ef59eebf3bc None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.043255 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ed3dc-9bfb-4db0-939e-3ef59eebf3bc None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.043529 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ed3dc-9bfb-4db0-939e-3ef59eebf3bc None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.044778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ed3dc-9bfb-4db0-939e-3ef59eebf3bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:36:51.045083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ed3dc-9bfb-4db0-939e-3ef59eebf3bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:51.046531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ed3dc-9bfb-4db0-939e-3ef59eebf3bc None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:51.059241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 17/32] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:36:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:51.060793 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfb5b7-c4fa-4cd0-8da2-490d0b39f50e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.061065 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfb5b7-c4fa-4cd0-8da2-490d0b39f50e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.061405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfb5b7-c4fa-4cd0-8da2-490d0b39f50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.081926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-decfb5b7-c4fa-4cd0-8da2-490d0b39f50e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:51.083204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-decfb5b7-c4fa-4cd0-8da2-490d0b39f50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:51.161629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 18/33] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:36:51.162778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfb5b7-c4fa-4cd0-8da2-490d0b39f50e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:51.239705 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:51.284236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfb5b7-c4fa-4cd0-8da2-490d0b39f50e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:51.321071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516bb68-de0c-4795-9b58-660409a61499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:51.321799 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516bb68-de0c-4795-9b58-660409a61499 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.322095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516bb68-de0c-4795-9b58-660409a61499 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.322374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516bb68-de0c-4795-9b58-660409a61499 None admin] PATH_INFO: `/v3/users/c14eb0bf645e49f887a8d288d37d1491` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.323536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516bb68-de0c-4795-9b58-660409a61499 None admin] RBAC: Authorizing `identity:get_user(user_id=c14eb0bf645e49f887a8d288d37d1491)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:51.325279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516bb68-de0c-4795-9b58-660409a61499 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:51.327400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 19/34] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:36:51 2022] GET /identity/v3/users/c14eb0bf645e49f887a8d288d37d1491 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:51.328330 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516bb68-de0c-4795-9b58-660409a61499 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:51.367155 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 16/35] 10.222.0.91 () {60 vars in 1141 bytes} [Thu Sep 8 13:36:50 2022] POST /identity/v3/projects => generated 301 bytes in 597 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:36:51.368973 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18691bb-d148-4e45-bb8b-05f2a3ce9150 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.369452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18691bb-d148-4e45-bb8b-05f2a3ce9150 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.369892 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18691bb-d148-4e45-bb8b-05f2a3ce9150 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.371368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 17/36] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:51.376829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18691bb-d148-4e45-bb8b-05f2a3ce9150 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:51.403868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:51.497535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18691bb-d148-4e45-bb8b-05f2a3ce9150 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:51.518598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516bb68-de0c-4795-9b58-660409a61499 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:51.539176 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a586e0-109f-4feb-905b-8ef91faf74f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:51.539740 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a586e0-109f-4feb-905b-8ef91faf74f5 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.540026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a586e0-109f-4feb-905b-8ef91faf74f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.540304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a586e0-109f-4feb-905b-8ef91faf74f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.541566 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a586e0-109f-4feb-905b-8ef91faf74f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:51.543390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a586e0-109f-4feb-905b-8ef91faf74f5 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:51.545448 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 18/37] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:36:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:51.546657 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0f861-a484-4472-952b-b24741c3b0a0 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.547118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0f861-a484-4472-952b-b24741c3b0a0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.547415 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0f861-a484-4472-952b-b24741c3b0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.560432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:51.569447 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-63e0f861-a484-4472-952b-b24741c3b0a0 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:51.571111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-63e0f861-a484-4472-952b-b24741c3b0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:51.578109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f412cae-4762-4195-a23b-490039727700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:51.578109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f412cae-4762-4195-a23b-490039727700 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.578109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f412cae-4762-4195-a23b-490039727700 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.578109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f412cae-4762-4195-a23b-490039727700 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.578491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f412cae-4762-4195-a23b-490039727700 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:51.578631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f412cae-4762-4195-a23b-490039727700 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:51.656189 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 19/38] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:36:51.657519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0f861-a484-4472-952b-b24741c3b0a0 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:51.668437 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:51.788436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0f861-a484-4472-952b-b24741c3b0a0 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:51.826123 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53bf5f-1484-4f32-9d85-5aa6bd41273f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:51.826647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53bf5f-1484-4f32-9d85-5aa6bd41273f None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.826962 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53bf5f-1484-4f32-9d85-5aa6bd41273f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.827239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53bf5f-1484-4f32-9d85-5aa6bd41273f None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.827900 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53bf5f-1484-4f32-9d85-5aa6bd41273f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:51.829267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53bf5f-1484-4f32-9d85-5aa6bd41273f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:51.839164 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 20/39] 10.222.0.91 () {60 vars in 1131 bytes} [Thu Sep 8 13:36:51 2022] POST /identity/v3/roles => generated 225 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:36:51.839164 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f412cae-4762-4195-a23b-490039727700 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:51.946872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 20/40] 10.222.0.91 () {60 vars in 1131 bytes} [Thu Sep 8 13:36:51 2022] POST /identity/v3/roles => generated 219 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:36:51.948346 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70db18b-de19-4f60-b531-5e6bb9fa0a5a None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.948639 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70db18b-de19-4f60-b531-5e6bb9fa0a5a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.949066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70db18b-de19-4f60-b531-5e6bb9fa0a5a None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.950741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 21/41] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:51.956551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70db18b-de19-4f60-b531-5e6bb9fa0a5a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:52.048893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:52.095857 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70db18b-de19-4f60-b531-5e6bb9fa0a5a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:52.097617 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f412cae-4762-4195-a23b-490039727700 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:52.135538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfacea2-09ea-4cf4-9f3a-8c4357ed1e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:52.136255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfacea2-09ea-4cf4-9f3a-8c4357ed1e34 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:52.136946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd873cb-94b7-443f-9e6d-07238b977614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:52.137063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfacea2-09ea-4cf4-9f3a-8c4357ed1e34 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:52.137063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfacea2-09ea-4cf4-9f3a-8c4357ed1e34 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:52.137187 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd873cb-94b7-443f-9e6d-07238b977614 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:52.137415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd873cb-94b7-443f-9e6d-07238b977614 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:52.137547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfacea2-09ea-4cf4-9f3a-8c4357ed1e34 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:52.138426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd873cb-94b7-443f-9e6d-07238b977614 None admin] PATH_INFO: `/v3/domains/default/users/c14eb0bf645e49f887a8d288d37d1491/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:52.138988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfacea2-09ea-4cf4-9f3a-8c4357ed1e34 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:52.139971 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd873cb-94b7-443f-9e6d-07238b977614 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c14eb0bf645e49f887a8d288d37d1491, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:52.141377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd873cb-94b7-443f-9e6d-07238b977614 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:52.232334 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 22/42] 10.222.0.91 () {60 vars in 1131 bytes} [Thu Sep 8 13:36:51 2022] POST /identity/v3/roles => generated 223 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:36:52.233819 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a9672-5075-44f8-9cba-f818a6f61473 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:52.234109 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a9672-5075-44f8-9cba-f818a6f61473 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:52.234383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a9672-5075-44f8-9cba-f818a6f61473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:52.254205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1d3a9672-5075-44f8-9cba-f818a6f61473 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:52.255497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1d3a9672-5075-44f8-9cba-f818a6f61473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:52.272039 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 21/43] 10.222.0.91 () {58 vars in 1361 bytes} [Thu Sep 8 13:36:51 2022] PUT /identity/v3/domains/default/users/c14eb0bf645e49f887a8d288d37d1491/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:36:52.341113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 23/44] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:36:52.614301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd873cb-94b7-443f-9e6d-07238b977614 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:52.662371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:52.717279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd873cb-94b7-443f-9e6d-07238b977614 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:52.752752 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a0d67-b249-41d8-bfb5-9dda8b4813dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:52.753306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a0d67-b249-41d8-bfb5-9dda8b4813dd None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:52.753588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a0d67-b249-41d8-bfb5-9dda8b4813dd None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:52.753865 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a0d67-b249-41d8-bfb5-9dda8b4813dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:52.755508 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a0d67-b249-41d8-bfb5-9dda8b4813dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:52.757994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a0d67-b249-41d8-bfb5-9dda8b4813dd None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:52.758170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 22/45] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:36:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:52.763726 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a9672-5075-44f8-9cba-f818a6f61473 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:52.821845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:52.864512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a9672-5075-44f8-9cba-f818a6f61473 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:52.903699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6910a-cdb6-4fa7-adf7-d6a52af45426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:52.904316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6910a-cdb6-4fa7-adf7-d6a52af45426 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:52.904608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6910a-cdb6-4fa7-adf7-d6a52af45426 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:52.904888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6910a-cdb6-4fa7-adf7-d6a52af45426 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:52.905546 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e6910a-cdb6-4fa7-adf7-d6a52af45426 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:52.907040 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e6910a-cdb6-4fa7-adf7-d6a52af45426 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:52.980911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 24/46] 10.222.0.91 () {60 vars in 1141 bytes} [Thu Sep 8 13:36:52 2022] POST /identity/v3/projects => generated 312 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:36:54.438149 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674b902-9298-40cd-af83-237224f16b2c None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:54.438702 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674b902-9298-40cd-af83-237224f16b2c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:54.439006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674b902-9298-40cd-af83-237224f16b2c None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:54.440578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 23/47] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:54.446114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00af0e4-30d1-4cd8-a67e-674aae2d9b43 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:54.446552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00af0e4-30d1-4cd8-a67e-674aae2d9b43 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:54.446785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00af0e4-30d1-4cd8-a67e-674aae2d9b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:54.466277 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e00af0e4-30d1-4cd8-a67e-674aae2d9b43 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:54.467394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e00af0e4-30d1-4cd8-a67e-674aae2d9b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:54.545797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 25/48] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:36:54.816071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674b902-9298-40cd-af83-237224f16b2c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:54.923546 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674b902-9298-40cd-af83-237224f16b2c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:54.940293 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:54.962296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e42a2-11b3-4fe0-9ca6-902ccc618b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:54.963011 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e42a2-11b3-4fe0-9ca6-902ccc618b9c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:54.963297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e42a2-11b3-4fe0-9ca6-902ccc618b9c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:54.963569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e42a2-11b3-4fe0-9ca6-902ccc618b9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:54.975908 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e42a2-11b3-4fe0-9ca6-902ccc618b9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:36:54.981639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-c63e42a2-11b3-4fe0-9ca6-902ccc618b9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:36:54.981639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:36:54.981639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:36:54.981639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:36:54.981639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:36:54.981639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:36:54.981639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:36:54.981639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:36:54.981639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:36:54.981639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:36:54.981639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:36:54.981639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:36:54.981639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:36:54.981639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:36:54.981639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:36:54.981639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:36:54.982672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:36:54.982672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:36:54.982672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:36:54.982672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:36:54.982672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:36:54.982672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:36:54.982672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:36:54.982672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:36:54.982672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:36:54.982672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:36:54.982672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:36:54.982672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:36:54.982672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:36:54.982672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:36:54.982672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:36:54.982672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:36:54.983768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:36:54.983768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:36:54.983768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:36:54.983768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:36:54.983768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:36:54.983768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:36:54.983768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:36:54.983768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:36:54.983768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:36:54.983768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:36:54.983768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:36:54.983768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:36:54.983768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 24/49] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:36:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:36:54.989082 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00af0e4-30d1-4cd8-a67e-674aae2d9b43 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:55.032534 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:55.098067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00af0e4-30d1-4cd8-a67e-674aae2d9b43 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:55.132389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a15d0-1e0a-4190-84e4-719d0c3a1679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:55.132998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a15d0-1e0a-4190-84e4-719d0c3a1679 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:55.133275 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a15d0-1e0a-4190-84e4-719d0c3a1679 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:55.133548 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a15d0-1e0a-4190-84e4-719d0c3a1679 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:55.134210 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a15d0-1e0a-4190-84e4-719d0c3a1679 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:36:55.134510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a15d0-1e0a-4190-84e4-719d0c3a1679 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:55.135930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a15d0-1e0a-4190-84e4-719d0c3a1679 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:55.149235 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 26/50] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:36:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:36:55.155203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e42a2-11b3-4fe0-9ca6-902ccc618b9c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:55.175573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e458c-ed87-4ae3-b0e1-3fa70388d466 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:55.175860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e458c-ed87-4ae3-b0e1-3fa70388d466 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:55.176141 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e458c-ed87-4ae3-b0e1-3fa70388d466 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:55.177613 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 27/51] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:36:55.182254 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a10a1-7abd-4956-8799-4fc0e082422f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:55.182540 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a10a1-7abd-4956-8799-4fc0e082422f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:55.182821 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a10a1-7abd-4956-8799-4fc0e082422f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:55.260472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e42a2-11b3-4fe0-9ca6-902ccc618b9c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:55.296952 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdf126-b968-469c-994d-a52d915560c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:55.297578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdf126-b968-469c-994d-a52d915560c2 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:55.297855 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdf126-b968-469c-994d-a52d915560c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:55.298125 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdf126-b968-469c-994d-a52d915560c2 None admin] PATH_INFO: `/v3/users/c14eb0bf645e49f887a8d288d37d1491` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:55.299269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cdf126-b968-469c-994d-a52d915560c2 None admin] RBAC: Authorizing `identity:get_user(user_id=c14eb0bf645e49f887a8d288d37d1491)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:55.300753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cdf126-b968-469c-994d-a52d915560c2 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:55.302522 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 25/52] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:36:55 2022] GET /identity/v3/users/c14eb0bf645e49f887a8d288d37d1491 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:55.304489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-741a10a1-7abd-4956-8799-4fc0e082422f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:55.305705 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-741a10a1-7abd-4956-8799-4fc0e082422f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:55.307883 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdf126-b968-469c-994d-a52d915560c2 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:55.350749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:55.376721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 28/53] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:36:55.407761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdf126-b968-469c-994d-a52d915560c2 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:55.441802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2951ca-fbf3-43c7-9451-3a878bd00bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:55.442677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2951ca-fbf3-43c7-9451-3a878bd00bbd None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:55.442776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2951ca-fbf3-43c7-9451-3a878bd00bbd None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:55.444174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2951ca-fbf3-43c7-9451-3a878bd00bbd None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:55.444445 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2951ca-fbf3-43c7-9451-3a878bd00bbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:55.445908 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2951ca-fbf3-43c7-9451-3a878bd00bbd None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:55.447563 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 26/54] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:36:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:36:55.454284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a10a1-7abd-4956-8799-4fc0e082422f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:55.489544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:55.559500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a10a1-7abd-4956-8799-4fc0e082422f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:55.593432 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3cc06f-004e-4b7b-8abd-b57fc9318f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:55.594085 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cc06f-004e-4b7b-8abd-b57fc9318f44 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:55.594365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cc06f-004e-4b7b-8abd-b57fc9318f44 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:55.594638 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cc06f-004e-4b7b-8abd-b57fc9318f44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:55.595400 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3cc06f-004e-4b7b-8abd-b57fc9318f44 None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.domain.id=default, user.id=c14eb0bf645e49f887a8d288d37d1491) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:36:55.595709 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3cc06f-004e-4b7b-8abd-b57fc9318f44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:55.597166 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3cc06f-004e-4b7b-8abd-b57fc9318f44 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:55.605804 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 29/55] 10.222.0.91 () {56 vars in 1322 bytes} [Thu Sep 8 13:36:55 2022] GET /identity/v3/role_assignments?user.id=c14eb0bf645e49f887a8d288d37d1491&scope.domain.id=default&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:55.645130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2951ca-fbf3-43c7-9451-3a878bd00bbd None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:55.750402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2951ca-fbf3-43c7-9451-3a878bd00bbd None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:55.759954 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:55.785588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f9c5e-0695-49e3-8086-2647f357a2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:55.786229 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f9c5e-0695-49e3-8086-2647f357a2e1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:55.786514 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f9c5e-0695-49e3-8086-2647f357a2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:55.786791 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f9c5e-0695-49e3-8086-2647f357a2e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:55.787949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f9c5e-0695-49e3-8086-2647f357a2e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:55.789528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f9c5e-0695-49e3-8086-2647f357a2e1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:55.791205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 27/56] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:36:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:55.797030 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3cc06f-004e-4b7b-8abd-b57fc9318f44 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:55.893988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:55.899516 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3cc06f-004e-4b7b-8abd-b57fc9318f44 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:55.933409 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a219da-c018-494a-a1db-8130759d742c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:55.934010 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a219da-c018-494a-a1db-8130759d742c None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:55.934295 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a219da-c018-494a-a1db-8130759d742c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:55.934615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a219da-c018-494a-a1db-8130759d742c None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:55.935300 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a219da-c018-494a-a1db-8130759d742c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:55.936763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a219da-c018-494a-a1db-8130759d742c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:56.043359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 30/57] 10.222.0.91 () {60 vars in 1140 bytes} [Thu Sep 8 13:36:55 2022] POST /identity/v3/projects => generated 298 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:36:58.198311 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d61e0-18d3-4c48-a37b-b63ffb8d9d5a None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:58.198828 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d61e0-18d3-4c48-a37b-b63ffb8d9d5a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:58.198939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d61e0-18d3-4c48-a37b-b63ffb8d9d5a None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:58.201095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 28/58] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:36:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:58.206319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917854ec-ceb2-4fb5-a841-4bc29974d722 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:58.206701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917854ec-ceb2-4fb5-a841-4bc29974d722 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:58.206878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917854ec-ceb2-4fb5-a841-4bc29974d722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:58.224760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-917854ec-ceb2-4fb5-a841-4bc29974d722 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:58.226216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-917854ec-ceb2-4fb5-a841-4bc29974d722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:58.289802 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 31/59] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:36:58.559358 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d61e0-18d3-4c48-a37b-b63ffb8d9d5a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:58.660364 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d61e0-18d3-4c48-a37b-b63ffb8d9d5a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:58.692735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e11434-aefa-48ae-a366-2cf9078dcab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:58.693352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e11434-aefa-48ae-a366-2cf9078dcab6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:58.693634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e11434-aefa-48ae-a366-2cf9078dcab6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:58.693907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e11434-aefa-48ae-a366-2cf9078dcab6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:58.695032 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e11434-aefa-48ae-a366-2cf9078dcab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:58.696490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e11434-aefa-48ae-a366-2cf9078dcab6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:58.698052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 29/60] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:36:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:58.704314 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917854ec-ceb2-4fb5-a841-4bc29974d722 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:58.780990 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:58.802189 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917854ec-ceb2-4fb5-a841-4bc29974d722 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:58.836319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500b1b3-8e96-412d-8702-da46b3eae09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:58.836921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500b1b3-8e96-412d-8702-da46b3eae09b None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:58.837219 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500b1b3-8e96-412d-8702-da46b3eae09b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:58.837498 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500b1b3-8e96-412d-8702-da46b3eae09b None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:58.838152 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500b1b3-8e96-412d-8702-da46b3eae09b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:58.839676 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500b1b3-8e96-412d-8702-da46b3eae09b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:58.956190 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 32/61] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:36:58 2022] POST /identity/v3/users => generated 275 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:02.380361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190952b-d4b9-4ee0-aa87-738e20758ca9 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:02.381429 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190952b-d4b9-4ee0-aa87-738e20758ca9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:02.381936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190952b-d4b9-4ee0-aa87-738e20758ca9 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:02.383520 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 30/62] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:02.389235 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bafe11-3e6e-4809-9fb8-97def8e31d4f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:02.390075 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bafe11-3e6e-4809-9fb8-97def8e31d4f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:02.390565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bafe11-3e6e-4809-9fb8-97def8e31d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:02.412873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc428e2-c4b5-43b4-8203-313507675b83 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:02.413197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc428e2-c4b5-43b4-8203-313507675b83 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:02.413432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc428e2-c4b5-43b4-8203-313507675b83 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:02.414785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 31/63] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:02.416994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-34bafe11-3e6e-4809-9fb8-97def8e31d4f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:02.418399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-34bafe11-3e6e-4809-9fb8-97def8e31d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:02.420483 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14aa05-d82d-4824-b877-72ad7f453332 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:02.420771 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14aa05-d82d-4824-b877-72ad7f453332 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:02.421047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14aa05-d82d-4824-b877-72ad7f453332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:02.458676 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ef14aa05-d82d-4824-b877-72ad7f453332 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:02.462438 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ef14aa05-d82d-4824-b877-72ad7f453332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:02.514659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 33/64] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:02.516003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd840c-7cb3-4eba-99eb-0074eda81c04 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:02.516450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd840c-7cb3-4eba-99eb-0074eda81c04 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:02.516869 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd840c-7cb3-4eba-99eb-0074eda81c04 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:02.518289 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 34/65] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:02.528439 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429ca71-2987-4cf2-ac21-7b55a9583cb8 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:02.529576 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429ca71-2987-4cf2-ac21-7b55a9583cb8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:02.530152 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429ca71-2987-4cf2-ac21-7b55a9583cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:02.546789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:02.557762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9429ca71-2987-4cf2-ac21-7b55a9583cb8 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:02.560614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9429ca71-2987-4cf2-ac21-7b55a9583cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:02.597346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 32/66] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:02.666199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:02.679925 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 35/67] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:02.800809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce3665-d82e-436f-bca4-7d000755405c None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:02.801476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce3665-d82e-436f-bca4-7d000755405c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:02.802168 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce3665-d82e-436f-bca4-7d000755405c None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:02.803801 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 33/68] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:02.809465 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14908f1d-0297-4c9f-b439-cfbd79327fdc None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:02.810052 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14908f1d-0297-4c9f-b439-cfbd79327fdc None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:02.810526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14908f1d-0297-4c9f-b439-cfbd79327fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:02.831272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-14908f1d-0297-4c9f-b439-cfbd79327fdc None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:02.832740 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-14908f1d-0297-4c9f-b439-cfbd79327fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:02.906978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af357c-2ada-4407-b8b8-805bfe04df47 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:02.907610 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af357c-2ada-4407-b8b8-805bfe04df47 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:02.908099 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af357c-2ada-4407-b8b8-805bfe04df47 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:02.909979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 34/69] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:02.917193 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 36/70] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:02.922931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5406f4-7b28-4ac7-8f3b-f2a0046e3f4e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:02.922931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5406f4-7b28-4ac7-8f3b-f2a0046e3f4e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:02.922931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5406f4-7b28-4ac7-8f3b-f2a0046e3f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:02.937504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14908f1d-0297-4c9f-b439-cfbd79327fdc None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:02.948951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-4b5406f4-7b28-4ac7-8f3b-f2a0046e3f4e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:02.951289 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4b5406f4-7b28-4ac7-8f3b-f2a0046e3f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:03.044290 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 35/71] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:03.045474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5406f4-7b28-4ac7-8f3b-f2a0046e3f4e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:03.050071 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14908f1d-0297-4c9f-b439-cfbd79327fdc None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:03.107728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:03.125426 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753f963-8e6d-4548-b926-184b0a717bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:03.126063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753f963-8e6d-4548-b926-184b0a717bc4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:03.126350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753f963-8e6d-4548-b926-184b0a717bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:03.126623 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753f963-8e6d-4548-b926-184b0a717bc4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:03.137470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753f963-8e6d-4548-b926-184b0a717bc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:03.142636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-c753f963-8e6d-4548-b926-184b0a717bc4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:03.142636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:03.142636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:03.142636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:03.142636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:03.142636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:03.142636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:03.142636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:03.142636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:03.142636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:03.142636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:03.142636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:03.142636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:03.142636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:03.142636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:03.142636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:03.143497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:03.143497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:03.143497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:03.143497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:03.143497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:03.143497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:03.143497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:03.143497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:03.143497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:03.143497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:03.143497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:03.143497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:03.143497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:03.143497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:03.143497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:03.143497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:03.144854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:03.144854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:03.144854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:03.144854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:03.144854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:03.144854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:03.144854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:03.144854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:03.144854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:03.144854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:03.144854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:03.144854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:03.144854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 37/72] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:37:03.145580 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753f963-8e6d-4548-b926-184b0a717bc4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:03.177816 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5406f4-7b28-4ac7-8f3b-f2a0046e3f4e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:03.181324 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:03.218633 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd846ae-2112-4016-9f12-ac051b0dda75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:03.219322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd846ae-2112-4016-9f12-ac051b0dda75 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:03.219606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd846ae-2112-4016-9f12-ac051b0dda75 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:03.219892 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd846ae-2112-4016-9f12-ac051b0dda75 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:03.230586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd846ae-2112-4016-9f12-ac051b0dda75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:03.235959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-efd846ae-2112-4016-9f12-ac051b0dda75 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:03.235959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:03.235959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:03.235959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:03.235959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:03.235959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:03.235959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:03.235959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:03.235959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:03.235959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:03.235959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:03.235959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:03.235959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:03.235959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:03.235959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:03.235959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:03.236776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:03.236776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:03.236776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:03.236776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:03.236776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:03.236776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:03.236776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:03.236776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:03.236776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:03.236776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:03.236776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:03.236776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:03.236776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:03.236776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:03.236776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:03.236776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:03.240015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:03.240015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:03.240015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:03.240015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:03.240015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:03.240015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:03.240015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:03.240015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:03.240015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:03.240015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:03.240015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:03.240015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:03.240015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 36/73] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:03.240015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd846ae-2112-4016-9f12-ac051b0dda75 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:03.253960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753f963-8e6d-4548-b926-184b0a717bc4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:03.290883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c40a5e-a4b3-479e-8139-52d53d90cd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:03.291537 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c40a5e-a4b3-479e-8139-52d53d90cd9b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:03.291824 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c40a5e-a4b3-479e-8139-52d53d90cd9b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:03.292107 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c40a5e-a4b3-479e-8139-52d53d90cd9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:03.302795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c40a5e-a4b3-479e-8139-52d53d90cd9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:03.307686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-f5c40a5e-a4b3-479e-8139-52d53d90cd9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:03.307686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:03.307686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:03.307686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:03.307686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:03.307686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:03.307686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:03.307686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:03.307686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:03.307686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:03.307686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:03.307686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:03.307686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:03.307686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:03.307686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:03.307686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:03.308683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:03.308683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:03.308683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:03.308683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:03.308683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:03.308683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:03.308683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:03.308683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:03.308683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:03.308683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:03.308683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:03.308683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:03.308683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:03.308683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:03.308683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:03.308683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:03.309431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:03.309431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:03.309431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:03.309431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:03.309431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:03.309431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:03.309431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:03.309431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:03.309431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:03.309431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:03.309431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:03.309431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:03.309431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 38/74] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:03.311890 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c40a5e-a4b3-479e-8139-52d53d90cd9b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:03.349712 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd846ae-2112-4016-9f12-ac051b0dda75 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:03.362317 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:03.383303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990673bc-1429-4ce2-ac46-3ee146fff47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:03.384494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990673bc-1429-4ce2-ac46-3ee146fff47a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:03.384775 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990673bc-1429-4ce2-ac46-3ee146fff47a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:03.385048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990673bc-1429-4ce2-ac46-3ee146fff47a None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:03.385715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990673bc-1429-4ce2-ac46-3ee146fff47a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:03.386020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990673bc-1429-4ce2-ac46-3ee146fff47a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:03.387456 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990673bc-1429-4ce2-ac46-3ee146fff47a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:03.395536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 37/75] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:03.399751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990673bc-1429-4ce2-ac46-3ee146fff47a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:03.416346 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c40a5e-a4b3-479e-8139-52d53d90cd9b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:03.427532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:03.450841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf12fef-6b4a-4a64-8812-a1bbdf8b8807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:03.451437 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf12fef-6b4a-4a64-8812-a1bbdf8b8807 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:03.451719 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf12fef-6b4a-4a64-8812-a1bbdf8b8807 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:03.451992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf12fef-6b4a-4a64-8812-a1bbdf8b8807 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:03.462164 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf12fef-6b4a-4a64-8812-a1bbdf8b8807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:03.467666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-bbf12fef-6b4a-4a64-8812-a1bbdf8b8807 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:03.467666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:03.467666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:03.467666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:03.467666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:03.467666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:03.467666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:03.467666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:03.467666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:03.467666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:03.467666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:03.467666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:03.467666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:03.467666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:03.467666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:03.467666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:03.468646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:03.468646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:03.468646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:03.468646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:03.468646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:03.468646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:03.468646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:03.468646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:03.468646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:03.468646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:03.468646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:03.468646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:03.468646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:03.468646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:03.468646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:03.468646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:03.469520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:03.469520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:03.469520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:03.469520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:03.469520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:03.469520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:03.469520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:03.469520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:03.469520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:03.469520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:03.469520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:03.469520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:03.469520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 39/76] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:03.470221 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf12fef-6b4a-4a64-8812-a1bbdf8b8807 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:03.502820 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990673bc-1429-4ce2-ac46-3ee146fff47a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:03.537631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211358b-9f62-4943-bf57-b8c150cee79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:03.538234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211358b-9f62-4943-bf57-b8c150cee79c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:03.538514 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211358b-9f62-4943-bf57-b8c150cee79c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:03.538787 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211358b-9f62-4943-bf57-b8c150cee79c None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:03.539476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211358b-9f62-4943-bf57-b8c150cee79c None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:03.539779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211358b-9f62-4943-bf57-b8c150cee79c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:03.541152 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211358b-9f62-4943-bf57-b8c150cee79c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:03.549839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 38/77] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:03.550676 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211358b-9f62-4943-bf57-b8c150cee79c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:03.572357 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf12fef-6b4a-4a64-8812-a1bbdf8b8807 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:03.574622 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:03.606389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197d769-526f-4f5f-8cfd-361e4417eaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:03.607600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197d769-526f-4f5f-8cfd-361e4417eaaa None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:03.607898 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197d769-526f-4f5f-8cfd-361e4417eaaa None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:03.608219 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197d769-526f-4f5f-8cfd-361e4417eaaa None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:03.608878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197d769-526f-4f5f-8cfd-361e4417eaaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:03.609193 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197d769-526f-4f5f-8cfd-361e4417eaaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:03.610584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197d769-526f-4f5f-8cfd-361e4417eaaa None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:03.618885 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 40/78] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:03.619692 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197d769-526f-4f5f-8cfd-361e4417eaaa None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:03.639495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:03.651123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211358b-9f62-4943-bf57-b8c150cee79c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:03.685135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62961dd0-43f7-490e-97dc-3c936c726de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:03.685759 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62961dd0-43f7-490e-97dc-3c936c726de8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:03.686040 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62961dd0-43f7-490e-97dc-3c936c726de8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:03.686311 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62961dd0-43f7-490e-97dc-3c936c726de8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:03.687468 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62961dd0-43f7-490e-97dc-3c936c726de8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:03.688905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62961dd0-43f7-490e-97dc-3c936c726de8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:03.690464 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 39/79] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:03.691241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62961dd0-43f7-490e-97dc-3c936c726de8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:03.720679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197d769-526f-4f5f-8cfd-361e4417eaaa None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:03.754293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f5b50-41cb-468d-bcd0-0f4bcf837e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:03.755000 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f5b50-41cb-468d-bcd0-0f4bcf837e31 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:03.755302 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f5b50-41cb-468d-bcd0-0f4bcf837e31 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:03.755578 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f5b50-41cb-468d-bcd0-0f4bcf837e31 None admin] PATH_INFO: `/v3/users/9f12e9e4aad240d09cdbf43f68f765de` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:03.771232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f5b50-41cb-468d-bcd0-0f4bcf837e31 None admin] RBAC: Authorizing `identity:get_user(user_id=9f12e9e4aad240d09cdbf43f68f765de)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:03.772699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398f5b50-41cb-468d-bcd0-0f4bcf837e31 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:03.774352 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 41/80] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:03 2022] GET /identity/v3/users/9f12e9e4aad240d09cdbf43f68f765de => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:03.775156 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f5b50-41cb-468d-bcd0-0f4bcf837e31 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:03.793894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62961dd0-43f7-490e-97dc-3c936c726de8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:03.800923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:03.827569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdde8e6-00f3-43b0-930c-f94e48953360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:03.828179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdde8e6-00f3-43b0-930c-f94e48953360 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:03.828465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdde8e6-00f3-43b0-930c-f94e48953360 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:03.828743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdde8e6-00f3-43b0-930c-f94e48953360 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:03.829431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdde8e6-00f3-43b0-930c-f94e48953360 None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:03.829738 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdde8e6-00f3-43b0-930c-f94e48953360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:03.831170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdde8e6-00f3-43b0-930c-f94e48953360 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:03.839205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 40/81] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:03.840004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdde8e6-00f3-43b0-930c-f94e48953360 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:03.864007 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:03.875238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f5b50-41cb-468d-bcd0-0f4bcf837e31 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:03.908936 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06d158-4568-46e3-ae93-ceed84fd3575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:03.909559 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06d158-4568-46e3-ae93-ceed84fd3575 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:03.909929 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06d158-4568-46e3-ae93-ceed84fd3575 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:03.910209 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06d158-4568-46e3-ae93-ceed84fd3575 None admin] PATH_INFO: `/v3/users/9f12e9e4aad240d09cdbf43f68f765de` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:03.911349 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a06d158-4568-46e3-ae93-ceed84fd3575 None admin] RBAC: Authorizing `identity:get_user(user_id=9f12e9e4aad240d09cdbf43f68f765de)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:03.912810 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a06d158-4568-46e3-ae93-ceed84fd3575 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:03.914550 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 42/82] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:03 2022] GET /identity/v3/users/9f12e9e4aad240d09cdbf43f68f765de => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:03.915251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06d158-4568-46e3-ae93-ceed84fd3575 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:03.948465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdde8e6-00f3-43b0-930c-f94e48953360 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:03.984230 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ea3af-f991-4eb2-8be4-e776ea6a2b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:03.984791 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ea3af-f991-4eb2-8be4-e776ea6a2b88 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:03.985077 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ea3af-f991-4eb2-8be4-e776ea6a2b88 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:03.985358 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ea3af-f991-4eb2-8be4-e776ea6a2b88 None admin] PATH_INFO: `/v3/users/c14eb0bf645e49f887a8d288d37d1491` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:03.986487 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ea3af-f991-4eb2-8be4-e776ea6a2b88 None admin] RBAC: Authorizing `identity:get_user(user_id=c14eb0bf645e49f887a8d288d37d1491)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:03.987979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ea3af-f991-4eb2-8be4-e776ea6a2b88 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:03.989709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 41/83] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:03 2022] GET /identity/v3/users/c14eb0bf645e49f887a8d288d37d1491 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:03.990398 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ea3af-f991-4eb2-8be4-e776ea6a2b88 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:04.011833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:04.020248 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06d158-4568-46e3-ae93-ceed84fd3575 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:04.053659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc0af5-2b38-415a-884e-1fd9975e7931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:04.054487 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc0af5-2b38-415a-884e-1fd9975e7931 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:04.054989 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc0af5-2b38-415a-884e-1fd9975e7931 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:04.055577 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc0af5-2b38-415a-884e-1fd9975e7931 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:04.056249 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc0af5-2b38-415a-884e-1fd9975e7931 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:04.057720 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc0af5-2b38-415a-884e-1fd9975e7931 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:04.073217 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:04.087463 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ea3af-f991-4eb2-8be4-e776ea6a2b88 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:04.120194 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0d48a-0fc0-4fef-b3d9-dd71acdc3b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:04.120792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0d48a-0fc0-4fef-b3d9-dd71acdc3b63 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:04.121072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0d48a-0fc0-4fef-b3d9-dd71acdc3b63 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:04.121342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0d48a-0fc0-4fef-b3d9-dd71acdc3b63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:04.145386 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 43/84] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:37:03 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:04.146772 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc0af5-2b38-415a-884e-1fd9975e7931 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:04.204934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0d48a-0fc0-4fef-b3d9-dd71acdc3b63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:04.206727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0d48a-0fc0-4fef-b3d9-dd71acdc3b63 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:04.267537 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc0af5-2b38-415a-884e-1fd9975e7931 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:04.304838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:04.307852 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1d8d1-bb3b-4474-82d3-e43a2ecec6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:04.308503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1d8d1-bb3b-4474-82d3-e43a2ecec6ea None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:04.308787 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1d8d1-bb3b-4474-82d3-e43a2ecec6ea None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:04.309070 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1d8d1-bb3b-4474-82d3-e43a2ecec6ea None admin] PATH_INFO: `/v3/users/9f12e9e4aad240d09cdbf43f68f765de` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:04.310278 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1d8d1-bb3b-4474-82d3-e43a2ecec6ea None admin] RBAC: Authorizing `identity:get_user(user_id=9f12e9e4aad240d09cdbf43f68f765de)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:04.311855 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1d8d1-bb3b-4474-82d3-e43a2ecec6ea None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:04.313457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 44/85] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:04 2022] GET /identity/v3/users/9f12e9e4aad240d09cdbf43f68f765de => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:04.318535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1d8d1-bb3b-4474-82d3-e43a2ecec6ea None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:04.325703 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 42/86] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:04.327786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0d48a-0fc0-4fef-b3d9-dd71acdc3b63 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:04.345615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:04.440753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0d48a-0fc0-4fef-b3d9-dd71acdc3b63 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:04.489313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08f501-a678-4c58-ab42-a8b928b5fc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:04.490861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08f501-a678-4c58-ab42-a8b928b5fc72 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:04.491028 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08f501-a678-4c58-ab42-a8b928b5fc72 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:04.491285 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08f501-a678-4c58-ab42-a8b928b5fc72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:04.499827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1d8d1-bb3b-4474-82d3-e43a2ecec6ea None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:04.548764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a85224-3dbe-4d09-849b-d0a05341d04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:04.549857 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a85224-3dbe-4d09-849b-d0a05341d04c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:04.550143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a85224-3dbe-4d09-849b-d0a05341d04c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:04.550417 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a85224-3dbe-4d09-849b-d0a05341d04c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:04.555430 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:04.571667 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08f501-a678-4c58-ab42-a8b928b5fc72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:04.573371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08f501-a678-4c58-ab42-a8b928b5fc72 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:04.645251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a85224-3dbe-4d09-849b-d0a05341d04c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:04.647121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a85224-3dbe-4d09-849b-d0a05341d04c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:04.670713 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:04.684225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 43/87] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:04.685677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08f501-a678-4c58-ab42-a8b928b5fc72 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:04.779856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:04.784914 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 45/88] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:04.789489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a85224-3dbe-4d09-849b-d0a05341d04c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:04.801563 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08f501-a678-4c58-ab42-a8b928b5fc72 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:04.855529 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c30d7f-8e1f-4317-9021-3249006a66cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:04.856150 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c30d7f-8e1f-4317-9021-3249006a66cc None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:04.856453 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c30d7f-8e1f-4317-9021-3249006a66cc None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:04.857660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c30d7f-8e1f-4317-9021-3249006a66cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:04.922192 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a85224-3dbe-4d09-849b-d0a05341d04c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:04.929693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c30d7f-8e1f-4317-9021-3249006a66cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:04.931527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c30d7f-8e1f-4317-9021-3249006a66cc None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:04.942836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:04.964713 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca048b3-947d-4409-9e22-17adc5d70010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:04.965774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca048b3-947d-4409-9e22-17adc5d70010 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:04.966521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca048b3-947d-4409-9e22-17adc5d70010 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:04.967307 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca048b3-947d-4409-9e22-17adc5d70010 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:04.968742 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca048b3-947d-4409-9e22-17adc5d70010 None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:04.970505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca048b3-947d-4409-9e22-17adc5d70010 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:04.972782 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 46/89] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:04 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:04.973849 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca048b3-947d-4409-9e22-17adc5d70010 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:05.035951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:05.058451 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 44/90] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:05.059261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c30d7f-8e1f-4317-9021-3249006a66cc None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:05.100772 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca048b3-947d-4409-9e22-17adc5d70010 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:05.137481 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1aecb-347a-4d00-acc4-b4bc22ef348f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:05.138514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1aecb-347a-4d00-acc4-b4bc22ef348f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:05.139037 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1aecb-347a-4d00-acc4-b4bc22ef348f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:05.139502 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1aecb-347a-4d00-acc4-b4bc22ef348f None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:05.140820 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d1aecb-347a-4d00-acc4-b4bc22ef348f None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:05.142528 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d1aecb-347a-4d00-acc4-b4bc22ef348f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:05.144692 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 47/91] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:04 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:05.145659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1aecb-347a-4d00-acc4-b4bc22ef348f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:05.192798 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c30d7f-8e1f-4317-9021-3249006a66cc None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:05.221707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:05.232209 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9f936-c79b-44f5-9c1e-d8988997a33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:05.237128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9f936-c79b-44f5-9c1e-d8988997a33a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:05.237128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9f936-c79b-44f5-9c1e-d8988997a33a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:05.237128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9f936-c79b-44f5-9c1e-d8988997a33a None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:05.237128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9f936-c79b-44f5-9c1e-d8988997a33a None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:05.237128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9f936-c79b-44f5-9c1e-d8988997a33a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:05.239161 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 45/92] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:05 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:05.240404 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9f936-c79b-44f5-9c1e-d8988997a33a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:05.282619 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1aecb-347a-4d00-acc4-b4bc22ef348f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:05.318395 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca4775-ea20-443f-a0dd-cb95821145e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:05.319131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:05.319608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca4775-ea20-443f-a0dd-cb95821145e3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:05.320080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca4775-ea20-443f-a0dd-cb95821145e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:05.320553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca4775-ea20-443f-a0dd-cb95821145e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:05.321460 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca4775-ea20-443f-a0dd-cb95821145e3 None admin] RBAC: Adding query filter params (role.id=1a62bc555eb345fc8679cfa677069fd7, scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72, user.id=9f12e9e4aad240d09cdbf43f68f765de) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:05.321969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca4775-ea20-443f-a0dd-cb95821145e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:05.323647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca4775-ea20-443f-a0dd-cb95821145e3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:05.330673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 48/93] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:05 2022] GET /identity/v3/role_assignments?user.id=9f12e9e4aad240d09cdbf43f68f765de&scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72&role.id=1a62bc555eb345fc8679cfa677069fd7 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:05.331908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca4775-ea20-443f-a0dd-cb95821145e3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:05.380713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9f936-c79b-44f5-9c1e-d8988997a33a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:05.419431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6909bc-8ff1-475f-af1b-a7199201dba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:05.420101 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6909bc-8ff1-475f-af1b-a7199201dba7 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:05.420416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6909bc-8ff1-475f-af1b-a7199201dba7 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:05.420690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6909bc-8ff1-475f-af1b-a7199201dba7 None admin] PATH_INFO: `/v3/projects/4d4683865db24d41b5be961336407802` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:05.421811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6909bc-8ff1-475f-af1b-a7199201dba7 None admin] RBAC: Authorizing `identity:get_project(project_id=4d4683865db24d41b5be961336407802)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:05.423289 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6909bc-8ff1-475f-af1b-a7199201dba7 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:05.424912 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 46/94] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:05 2022] GET /identity/v3/projects/4d4683865db24d41b5be961336407802 => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:05.425734 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6909bc-8ff1-475f-af1b-a7199201dba7 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:05.465923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca4775-ea20-443f-a0dd-cb95821145e3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:05.489457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:05.515911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb003b-1491-40a7-8cde-3c8dfcc8be2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:05.518481 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb003b-1491-40a7-8cde-3c8dfcc8be2a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:05.519917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb003b-1491-40a7-8cde-3c8dfcc8be2a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:05.520353 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb003b-1491-40a7-8cde-3c8dfcc8be2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:05.521331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb003b-1491-40a7-8cde-3c8dfcc8be2a None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72, user.id=c14eb0bf645e49f887a8d288d37d1491) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:05.523567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb003b-1491-40a7-8cde-3c8dfcc8be2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:05.526352 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb003b-1491-40a7-8cde-3c8dfcc8be2a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:05.540355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 49/95] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:05 2022] GET /identity/v3/role_assignments?user.id=c14eb0bf645e49f887a8d288d37d1491&scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:05.548066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb003b-1491-40a7-8cde-3c8dfcc8be2a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:05.619074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6909bc-8ff1-475f-af1b-a7199201dba7 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:05.666659 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:05.687047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8584369b-1b16-4be6-921a-8263211e7b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:05.690760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584369b-1b16-4be6-921a-8263211e7b9c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:05.690760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584369b-1b16-4be6-921a-8263211e7b9c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:05.690760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8584369b-1b16-4be6-921a-8263211e7b9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:05.690760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584369b-1b16-4be6-921a-8263211e7b9c None admin] RBAC: Adding query filter params (role.id=cb722313950945e5b147e994ec678cd1, scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72, user.id=9f12e9e4aad240d09cdbf43f68f765de) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:05.690760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584369b-1b16-4be6-921a-8263211e7b9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:05.691627 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8584369b-1b16-4be6-921a-8263211e7b9c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:05.704132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 47/96] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:05 2022] GET /identity/v3/role_assignments?user.id=9f12e9e4aad240d09cdbf43f68f765de&scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72&role.id=cb722313950945e5b147e994ec678cd1 => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:05.799179 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb003b-1491-40a7-8cde-3c8dfcc8be2a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:05.849106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee98c28-e206-4840-a500-535bc3d0fbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:05.849830 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee98c28-e206-4840-a500-535bc3d0fbf2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:05.850118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee98c28-e206-4840-a500-535bc3d0fbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:05.850390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee98c28-e206-4840-a500-535bc3d0fbf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:05.851174 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee98c28-e206-4840-a500-535bc3d0fbf2 None admin] RBAC: Adding query filter params (role.id=cb722313950945e5b147e994ec678cd1, scope.project.id=4d4683865db24d41b5be961336407802, user.id=9f12e9e4aad240d09cdbf43f68f765de) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:05.851558 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee98c28-e206-4840-a500-535bc3d0fbf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:05.853052 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee98c28-e206-4840-a500-535bc3d0fbf2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:05.860101 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 50/97] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:05 2022] GET /identity/v3/role_assignments?user.id=9f12e9e4aad240d09cdbf43f68f765de&scope.project.id=4d4683865db24d41b5be961336407802&role.id=cb722313950945e5b147e994ec678cd1 => generated 259 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:06.842746 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4411074-e845-430b-a3ee-16b253f2d208 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:06.843281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4411074-e845-430b-a3ee-16b253f2d208 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:06.843281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4411074-e845-430b-a3ee-16b253f2d208 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:06.844715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 48/98] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:06.850298 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ffd9c-b281-46a5-b290-024b04d55701 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:06.850696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ffd9c-b281-46a5-b290-024b04d55701 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:06.850860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ffd9c-b281-46a5-b290-024b04d55701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:06.874749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-5d1ffd9c-b281-46a5-b290-024b04d55701 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:06.875980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5d1ffd9c-b281-46a5-b290-024b04d55701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:06.887608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a3638-c7cc-409b-b62f-a63368639dab None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:06.887896 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a3638-c7cc-409b-b62f-a63368639dab None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:06.888177 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a3638-c7cc-409b-b62f-a63368639dab None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:06.889665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 49/99] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:06.897984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c43785-53a0-46ae-ac5b-4aa8c4e7aad3 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:06.898301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c43785-53a0-46ae-ac5b-4aa8c4e7aad3 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:06.898547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c43785-53a0-46ae-ac5b-4aa8c4e7aad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:06.923529 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b9c43785-53a0-46ae-ac5b-4aa8c4e7aad3 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:06.924738 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b9c43785-53a0-46ae-ac5b-4aa8c4e7aad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:06.930878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:06.975611 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 51/100] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:07.046241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 50/101] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:07.535099 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ffd9c-b281-46a5-b290-024b04d55701 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:07.557658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c43785-53a0-46ae-ac5b-4aa8c4e7aad3 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:07.682333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ffd9c-b281-46a5-b290-024b04d55701 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:07.728311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1558b3-560e-4b81-98f4-9e8f68fa4a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:07.728962 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1558b3-560e-4b81-98f4-9e8f68fa4a19 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:07.729243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1558b3-560e-4b81-98f4-9e8f68fa4a19 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:07.729513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1558b3-560e-4b81-98f4-9e8f68fa4a19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:07.731532 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1558b3-560e-4b81-98f4-9e8f68fa4a19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:07.733571 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1558b3-560e-4b81-98f4-9e8f68fa4a19 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:07.735292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 52/102] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:07.744601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1558b3-560e-4b81-98f4-9e8f68fa4a19 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:07.772035 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c43785-53a0-46ae-ac5b-4aa8c4e7aad3 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:07.814490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:07.850068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:07.862020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d96912-fe67-4f0c-b169-2fbd9eb63a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:07.862701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d96912-fe67-4f0c-b169-2fbd9eb63a43 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:07.863056 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d96912-fe67-4f0c-b169-2fbd9eb63a43 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:07.863285 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d96912-fe67-4f0c-b169-2fbd9eb63a43 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:07.889927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d96912-fe67-4f0c-b169-2fbd9eb63a43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:07.895446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-12d96912-fe67-4f0c-b169-2fbd9eb63a43 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:07.895446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:07.895446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:07.895446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:07.895446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:07.895446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:07.895446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:07.895446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:07.895446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:07.895446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:07.895446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:07.895446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:07.895446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:07.895446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:07.895446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:07.895446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:07.896298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:07.896298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:07.896298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:07.896298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:07.896298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:07.896298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:07.896298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:07.896298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:07.896298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:07.896298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:07.896298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:07.896298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:07.896298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:07.896298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:07.896298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:07.896298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:07.897607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:07.897607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:07.897607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:07.897607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:07.897607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:07.897607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:07.897607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:07.897607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:07.897607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:07.897607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:07.897607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:07.897607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:07.897607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 51/103] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:07.904308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d96912-fe67-4f0c-b169-2fbd9eb63a43 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:08.005146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1558b3-560e-4b81-98f4-9e8f68fa4a19 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:08.095324 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d96912-fe67-4f0c-b169-2fbd9eb63a43 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:08.101077 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa629d-0fcc-4b3e-a9fe-9edc44cdd5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:08.101889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa629d-0fcc-4b3e-a9fe-9edc44cdd5f2 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:08.102318 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa629d-0fcc-4b3e-a9fe-9edc44cdd5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:08.102740 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa629d-0fcc-4b3e-a9fe-9edc44cdd5f2 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:08.103580 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa629d-0fcc-4b3e-a9fe-9edc44cdd5f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:08.105198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa629d-0fcc-4b3e-a9fe-9edc44cdd5f2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:08.172996 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ecf5e-994d-4cbf-b935-3a675320ee95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:08.173645 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ecf5e-994d-4cbf-b935-3a675320ee95 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:08.173928 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ecf5e-994d-4cbf-b935-3a675320ee95 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:08.174204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ecf5e-994d-4cbf-b935-3a675320ee95 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:08.174878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ecf5e-994d-4cbf-b935-3a675320ee95 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:08.175208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ecf5e-994d-4cbf-b935-3a675320ee95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:08.176673 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ecf5e-994d-4cbf-b935-3a675320ee95 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:08.194390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 52/104] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:08.200235 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ecf5e-994d-4cbf-b935-3a675320ee95 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:08.229499 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 53/105] 10.222.0.91 () {60 vars in 1141 bytes} [Thu Sep 8 13:37:07 2022] POST /identity/v3/projects => generated 302 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:08.278277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:08.360908 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ecf5e-994d-4cbf-b935-3a675320ee95 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:08.362426 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036973a-736f-4aa8-8c9d-c56a55a8e4ce None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:08.362987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036973a-736f-4aa8-8c9d-c56a55a8e4ce None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:08.363544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036973a-736f-4aa8-8c9d-c56a55a8e4ce None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:08.366943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 54/106] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:08.371673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758e943-be28-4111-9d4a-bf46399b805c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:08.372793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758e943-be28-4111-9d4a-bf46399b805c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:08.372793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758e943-be28-4111-9d4a-bf46399b805c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:08.416095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b758e943-be28-4111-9d4a-bf46399b805c None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:08.417717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b758e943-be28-4111-9d4a-bf46399b805c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:08.478982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41042054-f241-4caa-a382-396fabc7e70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:08.479662 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41042054-f241-4caa-a382-396fabc7e70a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:08.480025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41042054-f241-4caa-a382-396fabc7e70a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:08.480312 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41042054-f241-4caa-a382-396fabc7e70a None admin] PATH_INFO: `/v3/users/cf0367416938434197b41766ae665b2b` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:08.517594 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41042054-f241-4caa-a382-396fabc7e70a None admin] RBAC: Authorizing `identity:get_user(user_id=cf0367416938434197b41766ae665b2b)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:08.519099 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41042054-f241-4caa-a382-396fabc7e70a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:08.522267 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 53/107] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:08 2022] GET /identity/v3/users/cf0367416938434197b41766ae665b2b => generated 289 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:08.528540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41042054-f241-4caa-a382-396fabc7e70a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:08.580527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 55/108] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:08.659592 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41042054-f241-4caa-a382-396fabc7e70a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:08.663283 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:08.697210 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7dcbf1-185a-40ce-aacc-5aa2ca3928aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:08.697875 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7dcbf1-185a-40ce-aacc-5aa2ca3928aa None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:08.698166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7dcbf1-185a-40ce-aacc-5aa2ca3928aa None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:08.698448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7dcbf1-185a-40ce-aacc-5aa2ca3928aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:08.779241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7dcbf1-185a-40ce-aacc-5aa2ca3928aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:08.780763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7dcbf1-185a-40ce-aacc-5aa2ca3928aa None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:08.893371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:08.903365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 54/109] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:08.910742 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758e943-be28-4111-9d4a-bf46399b805c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:08.926658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5aae6d-19db-4962-8ba5-302e6bc81e95 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:08.927052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5aae6d-19db-4962-8ba5-302e6bc81e95 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:08.927259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5aae6d-19db-4962-8ba5-302e6bc81e95 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:08.929057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 55/110] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:08.938966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ba0dd-a466-4cf4-afa7-4f262fdc8d0f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:08.939301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ba0dd-a466-4cf4-afa7-4f262fdc8d0f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:08.939525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ba0dd-a466-4cf4-afa7-4f262fdc8d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:08.975605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ac7ba0dd-a466-4cf4-afa7-4f262fdc8d0f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:08.977310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ac7ba0dd-a466-4cf4-afa7-4f262fdc8d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:08.980562 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:09.084392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758e943-be28-4111-9d4a-bf46399b805c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:09.122639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 56/111] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:09.126961 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ba0dd-a466-4cf4-afa7-4f262fdc8d0f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:09.130568 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363c951-bd98-4f36-ae22-86f8553b65b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:09.131631 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363c951-bd98-4f36-ae22-86f8553b65b8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:09.132060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363c951-bd98-4f36-ae22-86f8553b65b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:09.132475 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363c951-bd98-4f36-ae22-86f8553b65b8 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:09.133839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363c951-bd98-4f36-ae22-86f8553b65b8 None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:09.135647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363c951-bd98-4f36-ae22-86f8553b65b8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:09.137597 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 56/112] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:08 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:09.143712 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61282e9f-5aa0-408d-95cf-2a412c6617ee None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:09.144177 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61282e9f-5aa0-408d-95cf-2a412c6617ee None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:09.144581 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61282e9f-5aa0-408d-95cf-2a412c6617ee None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:09.146105 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 57/113] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:09.148133 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61282e9f-5aa0-408d-95cf-2a412c6617ee None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:09.275959 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:09.307370 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61282e9f-5aa0-408d-95cf-2a412c6617ee None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:09.332602 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ba0dd-a466-4cf4-afa7-4f262fdc8d0f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:09.364289 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:09.364756 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b0968-f350-4d03-8091-700c175d0cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:09.366244 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b0968-f350-4d03-8091-700c175d0cfa None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:09.366712 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b0968-f350-4d03-8091-700c175d0cfa None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:09.367184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b0968-f350-4d03-8091-700c175d0cfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:09.368099 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b0968-f350-4d03-8091-700c175d0cfa None admin] RBAC: Adding query filter params (role.id=d3fb1e3ec057480992710ad61b6cd1ed, scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72, user.id=cf0367416938434197b41766ae665b2b) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:09.368584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b0968-f350-4d03-8091-700c175d0cfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:09.370607 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b0968-f350-4d03-8091-700c175d0cfa None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:09.373270 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c29c4d-c78d-481c-aaba-bfacf95af881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:09.373905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c29c4d-c78d-481c-aaba-bfacf95af881 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:09.377540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c29c4d-c78d-481c-aaba-bfacf95af881 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:09.377540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c29c4d-c78d-481c-aaba-bfacf95af881 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:09.377960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 58/114] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:09 2022] GET /identity/v3/role_assignments?user.id=cf0367416938434197b41766ae665b2b&scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72&role.id=d3fb1e3ec057480992710ad61b6cd1ed => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:09.379340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad36708-cee6-49a2-babb-bcde87da65d4 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:09.379848 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad36708-cee6-49a2-babb-bcde87da65d4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:09.380296 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad36708-cee6-49a2-babb-bcde87da65d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:09.394763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c29c4d-c78d-481c-aaba-bfacf95af881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:09.394763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-47c29c4d-c78d-481c-aaba-bfacf95af881 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:09.394763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:09.394763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:09.394763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:09.394763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:09.394763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:09.394763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:09.394763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:09.394763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:09.394763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:09.394763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:09.394763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:09.394763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:09.395708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:09.396532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:09.396532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:09.396532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:09.396532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:09.396532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:09.396532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:09.396532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:09.396532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:09.396532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:09.396532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:09.396532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:09.396532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:09.396532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:09.396532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:09.396532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 57/115] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:09.397231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a1f9f-cf52-4ba9-8134-69008c609835 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:09.397231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a1f9f-cf52-4ba9-8134-69008c609835 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:09.397231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a1f9f-cf52-4ba9-8134-69008c609835 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:09.399191 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 58/116] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:09.404485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a1f9f-cf52-4ba9-8134-69008c609835 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:09.439360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9ad36708-cee6-49a2-babb-bcde87da65d4 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:09.440610 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9ad36708-cee6-49a2-babb-bcde87da65d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:09.559490 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 59/117] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:09.563147 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a34c0f-10f4-4f17-8c77-c368fd3072d7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:09.563464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a34c0f-10f4-4f17-8c77-c368fd3072d7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:09.563745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a34c0f-10f4-4f17-8c77-c368fd3072d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:09.584325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a1f9f-cf52-4ba9-8134-69008c609835 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:09.600289 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c4a34c0f-10f4-4f17-8c77-c368fd3072d7 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:09.601801 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c4a34c0f-10f4-4f17-8c77-c368fd3072d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:09.628461 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149f5d5-97b2-48d5-994c-adb1928fa2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:09.628991 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149f5d5-97b2-48d5-994c-adb1928fa2c3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:09.629396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149f5d5-97b2-48d5-994c-adb1928fa2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:09.629642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149f5d5-97b2-48d5-994c-adb1928fa2c3 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:09.630439 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149f5d5-97b2-48d5-994c-adb1928fa2c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:09.631011 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149f5d5-97b2-48d5-994c-adb1928fa2c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:09.632302 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149f5d5-97b2-48d5-994c-adb1928fa2c3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:09.643074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 59/118] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:09.644748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149f5d5-97b2-48d5-994c-adb1928fa2c3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:09.718171 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 60/119] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:09.719418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a34c0f-10f4-4f17-8c77-c368fd3072d7 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:09.738614 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:09.763032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:09.845395 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149f5d5-97b2-48d5-994c-adb1928fa2c3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:09.900066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a34c0f-10f4-4f17-8c77-c368fd3072d7 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:09.910736 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a6600-9f60-48fd-84d8-3543a48ec800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:09.911394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a6600-9f60-48fd-84d8-3543a48ec800 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:09.911676 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a6600-9f60-48fd-84d8-3543a48ec800 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:09.912016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a6600-9f60-48fd-84d8-3543a48ec800 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:09.927268 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a6600-9f60-48fd-84d8-3543a48ec800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:09.932646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-708a6600-9f60-48fd-84d8-3543a48ec800 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:09.932646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:09.932646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:09.932646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:09.932646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:09.932646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:09.932646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:09.932646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:09.932646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:09.932646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:09.932646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:09.932646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:09.932646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:09.932646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:09.932646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:09.932646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:09.933473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:09.933473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:09.933473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:09.933473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:09.933473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:09.933473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:09.933473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:09.933473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:09.933473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:09.933473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:09.933473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:09.933473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:09.933473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:09.933473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:09.933473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:09.933473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:09.934219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:09.934219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:09.934219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:09.934219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:09.934219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:09.934219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:09.934219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:09.934219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:09.934219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:09.934219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:09.934219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:09.934219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:09.934219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 60/120] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:09.944436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a6600-9f60-48fd-84d8-3543a48ec800 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:09.977925 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191cb9a2-0d0a-4fee-b426-92d0af2970d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:09.978553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cb9a2-0d0a-4fee-b426-92d0af2970d4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:09.978832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cb9a2-0d0a-4fee-b426-92d0af2970d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:09.979133 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191cb9a2-0d0a-4fee-b426-92d0af2970d4 None admin] PATH_INFO: `/v3/users/9f12e9e4aad240d09cdbf43f68f765de` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:09.980392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191cb9a2-0d0a-4fee-b426-92d0af2970d4 None admin] RBAC: Authorizing `identity:get_user(user_id=9f12e9e4aad240d09cdbf43f68f765de)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:09.983407 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191cb9a2-0d0a-4fee-b426-92d0af2970d4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:09.984999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 61/121] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:09 2022] GET /identity/v3/users/9f12e9e4aad240d09cdbf43f68f765de => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:09.991552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191cb9a2-0d0a-4fee-b426-92d0af2970d4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:10.149531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a6600-9f60-48fd-84d8-3543a48ec800 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:10.150201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:10.160503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:10.168343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191cb9a2-0d0a-4fee-b426-92d0af2970d4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:10.193520 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56caef25-8c15-4c50-b8a8-4104abcc740a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:10.194228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56caef25-8c15-4c50-b8a8-4104abcc740a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:10.194516 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56caef25-8c15-4c50-b8a8-4104abcc740a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:10.194787 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56caef25-8c15-4c50-b8a8-4104abcc740a None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:10.195699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56caef25-8c15-4c50-b8a8-4104abcc740a None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:10.196023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56caef25-8c15-4c50-b8a8-4104abcc740a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:10.198063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56caef25-8c15-4c50-b8a8-4104abcc740a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:10.208301 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cf132-8462-48ee-9f6e-9781c9b3c1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:10.209142 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cf132-8462-48ee-9f6e-9781c9b3c1b2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:10.209387 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 61/122] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:10.213875 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56caef25-8c15-4c50-b8a8-4104abcc740a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:10.214308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cf132-8462-48ee-9f6e-9781c9b3c1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:10.214775 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6cf132-8462-48ee-9f6e-9781c9b3c1b2 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:10.216260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6cf132-8462-48ee-9f6e-9781c9b3c1b2 None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:10.218411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6cf132-8462-48ee-9f6e-9781c9b3c1b2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:10.221624 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 62/123] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:09 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:10.222829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cf132-8462-48ee-9f6e-9781c9b3c1b2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:10.327322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56caef25-8c15-4c50-b8a8-4104abcc740a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:10.358755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6cf132-8462-48ee-9f6e-9781c9b3c1b2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:10.365942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f6735-f70c-46b1-9f76-e5a7dfbb1214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:10.366924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f6735-f70c-46b1-9f76-e5a7dfbb1214 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:10.367377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f6735-f70c-46b1-9f76-e5a7dfbb1214 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:10.368091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f6735-f70c-46b1-9f76-e5a7dfbb1214 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:10.379448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f6735-f70c-46b1-9f76-e5a7dfbb1214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:10.385308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-bd3f6735-f70c-46b1-9f76-e5a7dfbb1214 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:10.385308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:10.385308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:10.385308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:10.385308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:10.385308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:10.385308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:10.385308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:10.385308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:10.385308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:10.385308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:10.385308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:10.385308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:10.385308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:10.385308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:10.385308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:10.386172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:10.386172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:10.386172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:10.386172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:10.386172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:10.386172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:10.386172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:10.386172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:10.386172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:10.386172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:10.386172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:10.386172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:10.386172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:10.386172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:10.386172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:10.386172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:10.386933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:10.386933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:10.386933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:10.386933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:10.386933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:10.386933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:10.386933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:10.386933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:10.386933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:10.386933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:10.386933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:10.386933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:10.389451 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 62/124] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:10.390541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f6735-f70c-46b1-9f76-e5a7dfbb1214 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:10.399019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020eba9-55b9-4900-9691-723b7477a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:10.399944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020eba9-55b9-4900-9691-723b7477a667 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:10.400477 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020eba9-55b9-4900-9691-723b7477a667 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:10.400955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9020eba9-55b9-4900-9691-723b7477a667 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:10.413761 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9020eba9-55b9-4900-9691-723b7477a667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:10.419403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-9020eba9-55b9-4900-9691-723b7477a667 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:10.419403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:10.419403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:10.419403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:10.419403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:10.419403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:10.419403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:10.419403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:10.419403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:10.419403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:10.419403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:10.419403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:10.419403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:10.419403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:10.419403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:10.419403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:10.420831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:10.420831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:10.420831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:10.420831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:10.420831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:10.420831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:10.420831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:10.420831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:10.420831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:10.420831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:10.420831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:10.420831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:10.420831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:10.420831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:10.420831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:10.420831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:10.421608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:10.421608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:10.421608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:10.421608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:10.421608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:10.421608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:10.421608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:10.421608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:10.421608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:10.421608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:10.421608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:10.421608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:10.424000 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 63/125] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:10.429493 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:10.431200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020eba9-55b9-4900-9691-723b7477a667 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:10.448240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:10.513121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f6735-f70c-46b1-9f76-e5a7dfbb1214 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:10.548583 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b6a73-3fb7-4b6b-a135-d7b58c242272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:10.549622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b6a73-3fb7-4b6b-a135-d7b58c242272 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:10.550083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b6a73-3fb7-4b6b-a135-d7b58c242272 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:10.550527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b6a73-3fb7-4b6b-a135-d7b58c242272 None admin] PATH_INFO: `/v3/users/9f12e9e4aad240d09cdbf43f68f765de` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:10.551924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b6a73-3fb7-4b6b-a135-d7b58c242272 None admin] RBAC: Authorizing `identity:get_user(user_id=9f12e9e4aad240d09cdbf43f68f765de)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:10.552766 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9020eba9-55b9-4900-9691-723b7477a667 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:10.553765 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b6a73-3fb7-4b6b-a135-d7b58c242272 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:10.555583 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 63/126] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:10 2022] GET /identity/v3/users/9f12e9e4aad240d09cdbf43f68f765de => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:10.556551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b6a73-3fb7-4b6b-a135-d7b58c242272 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:10.589649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7b56f-1f29-4750-9079-3f73b5dc7d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:10.590307 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7b56f-1f29-4750-9079-3f73b5dc7d4e None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:10.590587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7b56f-1f29-4750-9079-3f73b5dc7d4e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:10.590863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7b56f-1f29-4750-9079-3f73b5dc7d4e None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72/users/9f12e9e4aad240d09cdbf43f68f765de/roles/1a62bc555eb345fc8679cfa677069fd7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:10.593199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7b56f-1f29-4750-9079-3f73b5dc7d4e None admin] RBAC: Authorizing `identity:create_grant(project_id=7fc04ff6a9f94d04af0a564c8a1bab72, user_id=9f12e9e4aad240d09cdbf43f68f765de, role_id=1a62bc555eb345fc8679cfa677069fd7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:10.594756 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7b56f-1f29-4750-9079-3f73b5dc7d4e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:10.658324 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:10.662354 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b6a73-3fb7-4b6b-a135-d7b58c242272 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:10.668726 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 64/127] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:10 2022] PUT /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72/users/9f12e9e4aad240d09cdbf43f68f765de/roles/1a62bc555eb345fc8679cfa677069fd7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:10.670386 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7b56f-1f29-4750-9079-3f73b5dc7d4e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:10.740347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2612a-ba31-4e55-b247-285793523ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:10.741880 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2612a-ba31-4e55-b247-285793523ebb None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:10.742370 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2612a-ba31-4e55-b247-285793523ebb None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:10.743932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2612a-ba31-4e55-b247-285793523ebb None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:10.744668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2612a-ba31-4e55-b247-285793523ebb None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:10.745829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2612a-ba31-4e55-b247-285793523ebb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:10.748776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a2612a-ba31-4e55-b247-285793523ebb None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:10.767494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 64/128] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:10.768398 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2612a-ba31-4e55-b247-285793523ebb None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:10.840245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:10.884948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7b56f-1f29-4750-9079-3f73b5dc7d4e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:10.925776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3bf29d-fbfb-49ef-ba99-f87a57c42548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:10.926476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bf29d-fbfb-49ef-ba99-f87a57c42548 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:10.926937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bf29d-fbfb-49ef-ba99-f87a57c42548 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:10.927281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3bf29d-fbfb-49ef-ba99-f87a57c42548 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:10.928095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3bf29d-fbfb-49ef-ba99-f87a57c42548 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:10.928410 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3bf29d-fbfb-49ef-ba99-f87a57c42548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:10.929812 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3bf29d-fbfb-49ef-ba99-f87a57c42548 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:10.938213 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2612a-ba31-4e55-b247-285793523ebb None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:10.939822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 65/129] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:10.940649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3bf29d-fbfb-49ef-ba99-f87a57c42548 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:10.977978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa287bf6-f8e7-4045-bd2d-dc314a2e7c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:10.978712 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa287bf6-f8e7-4045-bd2d-dc314a2e7c56 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:10.979030 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa287bf6-f8e7-4045-bd2d-dc314a2e7c56 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:10.979309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa287bf6-f8e7-4045-bd2d-dc314a2e7c56 None admin] PATH_INFO: `/v3/projects/4d4683865db24d41b5be961336407802` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:10.980473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa287bf6-f8e7-4045-bd2d-dc314a2e7c56 None admin] RBAC: Authorizing `identity:get_project(project_id=4d4683865db24d41b5be961336407802)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:10.982165 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa287bf6-f8e7-4045-bd2d-dc314a2e7c56 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:10.984243 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 65/130] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:10 2022] GET /identity/v3/projects/4d4683865db24d41b5be961336407802 => generated 312 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:10.985269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa287bf6-f8e7-4045-bd2d-dc314a2e7c56 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:11.021216 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:11.065235 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3bf29d-fbfb-49ef-ba99-f87a57c42548 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:11.111906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa4fec-f348-49f5-9aa7-ae0a644829aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:11.112806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa4fec-f348-49f5-9aa7-ae0a644829aa None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:11.113468 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa4fec-f348-49f5-9aa7-ae0a644829aa None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:11.113756 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa4fec-f348-49f5-9aa7-ae0a644829aa None admin] PATH_INFO: `/v3/users/9f12e9e4aad240d09cdbf43f68f765de` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:11.115214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa4fec-f348-49f5-9aa7-ae0a644829aa None admin] RBAC: Authorizing `identity:get_user(user_id=9f12e9e4aad240d09cdbf43f68f765de)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:11.116075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa287bf6-f8e7-4045-bd2d-dc314a2e7c56 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:11.116686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa4fec-f348-49f5-9aa7-ae0a644829aa None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:11.119362 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 66/131] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:10 2022] GET /identity/v3/users/9f12e9e4aad240d09cdbf43f68f765de => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:11.120140 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa4fec-f348-49f5-9aa7-ae0a644829aa None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:11.126308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:11.158396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0fc0e-3679-4d98-bc37-12f39bdfdad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:11.159244 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0fc0e-3679-4d98-bc37-12f39bdfdad3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:11.159526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0fc0e-3679-4d98-bc37-12f39bdfdad3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:11.160115 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0fc0e-3679-4d98-bc37-12f39bdfdad3 None admin] PATH_INFO: `/v3/users/c14eb0bf645e49f887a8d288d37d1491` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:11.161566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0fc0e-3679-4d98-bc37-12f39bdfdad3 None admin] RBAC: Authorizing `identity:get_user(user_id=c14eb0bf645e49f887a8d288d37d1491)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:11.163271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0fc0e-3679-4d98-bc37-12f39bdfdad3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:11.165643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 66/132] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:10 2022] GET /identity/v3/users/c14eb0bf645e49f887a8d288d37d1491 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:11.166418 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0fc0e-3679-4d98-bc37-12f39bdfdad3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:11.251365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa4fec-f348-49f5-9aa7-ae0a644829aa None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:11.273552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:11.280927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0fc0e-3679-4d98-bc37-12f39bdfdad3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:11.290708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4a776-2b4f-46e3-bd9d-a58314523cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:11.298805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4a776-2b4f-46e3-bd9d-a58314523cc3 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:11.298805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4a776-2b4f-46e3-bd9d-a58314523cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:11.298805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4a776-2b4f-46e3-bd9d-a58314523cc3 None admin] PATH_INFO: `/v3/projects/4d4683865db24d41b5be961336407802/users/9f12e9e4aad240d09cdbf43f68f765de/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:11.298805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4a776-2b4f-46e3-bd9d-a58314523cc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d4683865db24d41b5be961336407802, user_id=9f12e9e4aad240d09cdbf43f68f765de, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:11.298805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4a776-2b4f-46e3-bd9d-a58314523cc3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:11.318801 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d7bf4-9349-4ef6-9e37-6d5674192306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:11.319467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d7bf4-9349-4ef6-9e37-6d5674192306 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:11.319746 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d7bf4-9349-4ef6-9e37-6d5674192306 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:11.320020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d7bf4-9349-4ef6-9e37-6d5674192306 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:11.321280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d7bf4-9349-4ef6-9e37-6d5674192306 None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:11.322734 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d7bf4-9349-4ef6-9e37-6d5674192306 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:11.324664 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 67/133] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:11 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:11.328161 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977e831-5581-4046-b441-aafced17c535 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:11.328796 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977e831-5581-4046-b441-aafced17c535 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:11.329343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977e831-5581-4046-b441-aafced17c535 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:11.331011 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 68/134] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:11.331906 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977e831-5581-4046-b441-aafced17c535 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:11.440952 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977e831-5581-4046-b441-aafced17c535 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:11.483706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d566a34-0842-434b-bc86-e2205272ae26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:11.484433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d566a34-0842-434b-bc86-e2205272ae26 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:11.484713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d566a34-0842-434b-bc86-e2205272ae26 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:11.484993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d566a34-0842-434b-bc86-e2205272ae26 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:11.486218 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d566a34-0842-434b-bc86-e2205272ae26 None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:11.487778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d566a34-0842-434b-bc86-e2205272ae26 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:11.495677 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 67/135] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:11 2022] PUT /identity/v3/projects/4d4683865db24d41b5be961336407802/users/9f12e9e4aad240d09cdbf43f68f765de/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:11.496309 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4a776-2b4f-46e3-bd9d-a58314523cc3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:11.505642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 69/136] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:11 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:11.505810 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e13f49-29f1-4d62-8af1-07e03630b360 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:11.506009 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e13f49-29f1-4d62-8af1-07e03630b360 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:11.506269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e13f49-29f1-4d62-8af1-07e03630b360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:11.552135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-10e13f49-29f1-4d62-8af1-07e03630b360 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:11.553661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-10e13f49-29f1-4d62-8af1-07e03630b360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:11.581871 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:11.612834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:11.635319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4a776-2b4f-46e3-bd9d-a58314523cc3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:11.681782 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bbbe2-0442-4b57-96ea-54cafd6efb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:11.683095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bbbe2-0442-4b57-96ea-54cafd6efb93 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:11.683801 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bbbe2-0442-4b57-96ea-54cafd6efb93 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:11.684283 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bbbe2-0442-4b57-96ea-54cafd6efb93 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72/users/9f12e9e4aad240d09cdbf43f68f765de/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:11.687512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bbbe2-0442-4b57-96ea-54cafd6efb93 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fc04ff6a9f94d04af0a564c8a1bab72, user_id=9f12e9e4aad240d09cdbf43f68f765de, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:11.689509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bbbe2-0442-4b57-96ea-54cafd6efb93 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:11.710467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 70/137] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:11.711750 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e13f49-29f1-4d62-8af1-07e03630b360 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:11.799185 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 68/138] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:11 2022] PUT /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72/users/9f12e9e4aad240d09cdbf43f68f765de/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:11.867536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e13f49-29f1-4d62-8af1-07e03630b360 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:11.906776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb86b61-8d3f-4f96-92fe-7c80bf60e3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:11.907850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb86b61-8d3f-4f96-92fe-7c80bf60e3ac None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:11.908334 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb86b61-8d3f-4f96-92fe-7c80bf60e3ac None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:11.908806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb86b61-8d3f-4f96-92fe-7c80bf60e3ac None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72/users/c14eb0bf645e49f887a8d288d37d1491/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:11.911106 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb86b61-8d3f-4f96-92fe-7c80bf60e3ac None admin] RBAC: Authorizing `identity:create_grant(project_id=7fc04ff6a9f94d04af0a564c8a1bab72, user_id=c14eb0bf645e49f887a8d288d37d1491, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:11.912872 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb86b61-8d3f-4f96-92fe-7c80bf60e3ac None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:11.929817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033ea42-8ab5-4aae-b200-33d6e24dada5 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:11.936163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033ea42-8ab5-4aae-b200-33d6e24dada5 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:11.936163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033ea42-8ab5-4aae-b200-33d6e24dada5 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:11.936163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 69/139] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:11.938761 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd3ddf-5c8d-40e5-b607-c559d0b648aa None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:11.939116 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd3ddf-5c8d-40e5-b607-c559d0b648aa None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:11.939442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd3ddf-5c8d-40e5-b607-c559d0b648aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:11.960784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-6edd3ddf-5c8d-40e5-b607-c559d0b648aa None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:11.961980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6edd3ddf-5c8d-40e5-b607-c559d0b648aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:12.021037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 71/140] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:11 2022] PUT /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72/users/c14eb0bf645e49f887a8d288d37d1491/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:12.071676 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 70/141] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:12.094076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb86b61-8d3f-4f96-92fe-7c80bf60e3ac None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:12.128154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:12.228544 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb86b61-8d3f-4f96-92fe-7c80bf60e3ac None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:12.274719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8879601-0a9a-4edf-923d-e19f662d1a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:12.275404 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8879601-0a9a-4edf-923d-e19f662d1a11 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:12.275676 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8879601-0a9a-4edf-923d-e19f662d1a11 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:12.275947 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8879601-0a9a-4edf-923d-e19f662d1a11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:12.277131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8879601-0a9a-4edf-923d-e19f662d1a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:12.278600 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8879601-0a9a-4edf-923d-e19f662d1a11 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:12.280786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 72/142] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:12.286797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd3ddf-5c8d-40e5-b607-c559d0b648aa None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:12.354763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:12.411345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd3ddf-5c8d-40e5-b607-c559d0b648aa None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:12.451251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c1571-e8e2-4166-9893-f020a10dbc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:12.451857 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c1571-e8e2-4166-9893-f020a10dbc56 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:12.452137 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c1571-e8e2-4166-9893-f020a10dbc56 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:12.452411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c1571-e8e2-4166-9893-f020a10dbc56 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:12.453069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c1571-e8e2-4166-9893-f020a10dbc56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:12.454513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c1571-e8e2-4166-9893-f020a10dbc56 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:12.539748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8879601-0a9a-4edf-923d-e19f662d1a11 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:12.631588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 71/143] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:37:12 2022] POST /identity/v3/users => generated 283 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:12.666058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:12.681136 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8879601-0a9a-4edf-923d-e19f662d1a11 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:12.724158 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2240f34-b4e9-4955-96b2-aaf8a12b515b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:12.724816 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2240f34-b4e9-4955-96b2-aaf8a12b515b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:12.725087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2240f34-b4e9-4955-96b2-aaf8a12b515b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:12.725358 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2240f34-b4e9-4955-96b2-aaf8a12b515b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:12.738098 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2240f34-b4e9-4955-96b2-aaf8a12b515b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:12.743126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-c2240f34-b4e9-4955-96b2-aaf8a12b515b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:12.743126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:12.743126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:12.743126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:12.743126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:12.743126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:12.743126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:12.743126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:12.743126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:12.743126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:12.743126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:12.743126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:12.743126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:12.743126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:12.743126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:12.743126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:12.745236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:12.745236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:12.745236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:12.745236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:12.745236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:12.745236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:12.745236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:12.745236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:12.745236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:12.745236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:12.745236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:12.745236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:12.745236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:12.745236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:12.745236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:12.745236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:12.747206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:12.747206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:12.747206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:12.747206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:12.747206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:12.747206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:12.747206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:12.747206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:12.747206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:12.747206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:12.747206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:12.747206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:12.747206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 73/144] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:12.752786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c1571-e8e2-4166-9893-f020a10dbc56 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:12.912416 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c1571-e8e2-4166-9893-f020a10dbc56 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:12.928018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:12.961027 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db10026-25bd-49c5-b238-4d2503231a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:12.961649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db10026-25bd-49c5-b238-4d2503231a52 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:12.961928 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db10026-25bd-49c5-b238-4d2503231a52 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:12.962199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db10026-25bd-49c5-b238-4d2503231a52 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:12.962871 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db10026-25bd-49c5-b238-4d2503231a52 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:12.963202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db10026-25bd-49c5-b238-4d2503231a52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:12.964642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db10026-25bd-49c5-b238-4d2503231a52 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:12.980090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 72/145] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:12.985751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2240f34-b4e9-4955-96b2-aaf8a12b515b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:13.105134 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2240f34-b4e9-4955-96b2-aaf8a12b515b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:13.147733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c127388-5ad1-4fcc-a413-b2a51d9dae34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:13.148754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c127388-5ad1-4fcc-a413-b2a51d9dae34 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:13.149214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c127388-5ad1-4fcc-a413-b2a51d9dae34 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:13.149675 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c127388-5ad1-4fcc-a413-b2a51d9dae34 None admin] PATH_INFO: `/v3/users/cf0367416938434197b41766ae665b2b` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:13.151016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c127388-5ad1-4fcc-a413-b2a51d9dae34 None admin] RBAC: Authorizing `identity:get_user(user_id=cf0367416938434197b41766ae665b2b)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:13.152658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c127388-5ad1-4fcc-a413-b2a51d9dae34 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:13.155022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 74/146] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:12 2022] GET /identity/v3/users/cf0367416938434197b41766ae665b2b => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:13.160664 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db10026-25bd-49c5-b238-4d2503231a52 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:13.269675 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db10026-25bd-49c5-b238-4d2503231a52 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:13.311608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd2efb-f65b-4768-8b18-c7862d199810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:13.312995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd2efb-f65b-4768-8b18-c7862d199810 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:13.313464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd2efb-f65b-4768-8b18-c7862d199810 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:13.313902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd2efb-f65b-4768-8b18-c7862d199810 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:13.315253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd2efb-f65b-4768-8b18-c7862d199810 None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:13.316878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd2efb-f65b-4768-8b18-c7862d199810 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:13.318694 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 73/147] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:13 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:13.324525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c127388-5ad1-4fcc-a413-b2a51d9dae34 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:13.362225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:13.490728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf586ff9-0a39-46a8-81be-66eca83f8568 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:13.491104 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf586ff9-0a39-46a8-81be-66eca83f8568 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:13.491378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf586ff9-0a39-46a8-81be-66eca83f8568 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:13.493018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c127388-5ad1-4fcc-a413-b2a51d9dae34 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:13.493131 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 74/148] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:13.498064 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add31ffa-3cf1-4c29-9e98-978c01ea681d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:13.498344 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add31ffa-3cf1-4c29-9e98-978c01ea681d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:13.498614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add31ffa-3cf1-4c29-9e98-978c01ea681d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:13.531216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-add31ffa-3cf1-4c29-9e98-978c01ea681d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:13.532331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-add31ffa-3cf1-4c29-9e98-978c01ea681d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:13.560216 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c23b9-6f15-4fac-8cc3-a760d16d6ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:13.560985 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c23b9-6f15-4fac-8cc3-a760d16d6ac3 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:13.561261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c23b9-6f15-4fac-8cc3-a760d16d6ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:13.562489 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c23b9-6f15-4fac-8cc3-a760d16d6ac3 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72/users/cf0367416938434197b41766ae665b2b/roles/d3fb1e3ec057480992710ad61b6cd1ed` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:13.565551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c23b9-6f15-4fac-8cc3-a760d16d6ac3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fc04ff6a9f94d04af0a564c8a1bab72, user_id=cf0367416938434197b41766ae665b2b, role_id=d3fb1e3ec057480992710ad61b6cd1ed)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:13.568404 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c23b9-6f15-4fac-8cc3-a760d16d6ac3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:13.591457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:13.629299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 75/149] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:13.701994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 75/150] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:13 2022] PUT /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72/users/cf0367416938434197b41766ae665b2b/roles/d3fb1e3ec057480992710ad61b6cd1ed => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:13.987867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add31ffa-3cf1-4c29-9e98-978c01ea681d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:14.107445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add31ffa-3cf1-4c29-9e98-978c01ea681d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:14.149995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb836f3-10ab-4c67-8349-520562b2db0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:14.150644 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb836f3-10ab-4c67-8349-520562b2db0f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:14.150991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb836f3-10ab-4c67-8349-520562b2db0f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:14.151220 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb836f3-10ab-4c67-8349-520562b2db0f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:14.164178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb836f3-10ab-4c67-8349-520562b2db0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:14.168960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-1eb836f3-10ab-4c67-8349-520562b2db0f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:14.168960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:14.168960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:14.168960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:14.168960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:14.168960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:14.168960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:14.168960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:14.168960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:14.168960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:14.168960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:14.168960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:14.168960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:14.168960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:14.168960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:14.168960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:14.169817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:14.169817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:14.169817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:14.169817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:14.169817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:14.169817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:14.169817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:14.169817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:14.169817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:14.169817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:14.169817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:14.169817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:14.169817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:14.169817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:14.169817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:14.169817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:14.170601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:14.170601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:14.170601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:14.170601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:14.170601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:14.170601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:14.170601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:14.170601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:14.170601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:14.170601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:14.170601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:14.170601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:14.174834 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 76/151] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:14.180018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c23b9-6f15-4fac-8cc3-a760d16d6ac3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:14.307353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:14.346000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c23b9-6f15-4fac-8cc3-a760d16d6ac3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:14.392115 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077555b3-d503-4722-99fc-d9557e480cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:14.392730 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077555b3-d503-4722-99fc-d9557e480cce None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:14.393010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077555b3-d503-4722-99fc-d9557e480cce None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:14.393284 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077555b3-d503-4722-99fc-d9557e480cce None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:14.393950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077555b3-d503-4722-99fc-d9557e480cce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:14.394252 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077555b3-d503-4722-99fc-d9557e480cce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:14.395664 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077555b3-d503-4722-99fc-d9557e480cce None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:14.411706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 76/152] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:14.416788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb836f3-10ab-4c67-8349-520562b2db0f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:14.470579 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:14.558060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb836f3-10ab-4c67-8349-520562b2db0f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:14.595893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f416b11-a604-4653-81ea-01193b1c38e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:14.596528 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f416b11-a604-4653-81ea-01193b1c38e3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:14.596806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f416b11-a604-4653-81ea-01193b1c38e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:14.597075 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f416b11-a604-4653-81ea-01193b1c38e3 None admin] PATH_INFO: `/v3/users/9f12e9e4aad240d09cdbf43f68f765de` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:14.598203 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f416b11-a604-4653-81ea-01193b1c38e3 None admin] RBAC: Authorizing `identity:get_user(user_id=9f12e9e4aad240d09cdbf43f68f765de)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:14.599813 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f416b11-a604-4653-81ea-01193b1c38e3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:14.601785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 77/153] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:14 2022] GET /identity/v3/users/9f12e9e4aad240d09cdbf43f68f765de => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:14.607258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077555b3-d503-4722-99fc-d9557e480cce None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:14.740340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077555b3-d503-4722-99fc-d9557e480cce None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:14.775656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87da55-98e1-47a7-a61c-b72384c3a3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:14.776273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87da55-98e1-47a7-a61c-b72384c3a3b1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:14.776554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87da55-98e1-47a7-a61c-b72384c3a3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:14.776824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87da55-98e1-47a7-a61c-b72384c3a3b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:14.790068 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:14.842324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05728a9a-e6ce-473e-aa96-cb9e06451db5 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:14.842695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05728a9a-e6ce-473e-aa96-cb9e06451db5 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:14.843993 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05728a9a-e6ce-473e-aa96-cb9e06451db5 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:14.846995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 78/154] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:14.854289 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13a1b7-d875-4d7d-bad0-997641d44722 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:14.854593 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13a1b7-d875-4d7d-bad0-997641d44722 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:14.856968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87da55-98e1-47a7-a61c-b72384c3a3b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:14.857450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13a1b7-d875-4d7d-bad0-997641d44722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:14.859510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87da55-98e1-47a7-a61c-b72384c3a3b1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:14.904237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8d13a1b7-d875-4d7d-bad0-997641d44722 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:14.905463 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8d13a1b7-d875-4d7d-bad0-997641d44722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:14.984001 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 77/155] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:14.989191 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87da55-98e1-47a7-a61c-b72384c3a3b1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.013273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 79/156] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:15.057706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:15.162770 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87da55-98e1-47a7-a61c-b72384c3a3b1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:15.213928 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b51b4-8f4b-4265-a192-276417107642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:15.214542 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b51b4-8f4b-4265-a192-276417107642 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.214837 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b51b4-8f4b-4265-a192-276417107642 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.215183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b51b4-8f4b-4265-a192-276417107642 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.218293 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b51b4-8f4b-4265-a192-276417107642 None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:15.219793 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b51b4-8f4b-4265-a192-276417107642 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:15.221428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 78/157] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:14 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:15.227569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13a1b7-d875-4d7d-bad0-997641d44722 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.270537 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:15.363172 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13a1b7-d875-4d7d-bad0-997641d44722 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:15.387962 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b51b4-8f4b-4265-a192-276417107642 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.403907 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee36d1-cbd3-4948-9073-4069be83fe24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:15.405350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee36d1-cbd3-4948-9073-4069be83fe24 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.405634 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee36d1-cbd3-4948-9073-4069be83fe24 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.405995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee36d1-cbd3-4948-9073-4069be83fe24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.406854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee36d1-cbd3-4948-9073-4069be83fe24 None admin] RBAC: Adding query filter params (role.id=1a62bc555eb345fc8679cfa677069fd7, scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72, user.id=9f12e9e4aad240d09cdbf43f68f765de) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:15.406991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee36d1-cbd3-4948-9073-4069be83fe24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:15.408428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee36d1-cbd3-4948-9073-4069be83fe24 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:15.414758 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 80/158] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:15 2022] GET /identity/v3/role_assignments?user.id=9f12e9e4aad240d09cdbf43f68f765de&scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72&role.id=1a62bc555eb345fc8679cfa677069fd7 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:15.576617 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b51b4-8f4b-4265-a192-276417107642 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:15.595053 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:15.631071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bccda-779b-42c7-9cee-829867c0b573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:15.633098 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bccda-779b-42c7-9cee-829867c0b573 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.633641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bccda-779b-42c7-9cee-829867c0b573 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.634119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bccda-779b-42c7-9cee-829867c0b573 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.651047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bccda-779b-42c7-9cee-829867c0b573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:15.657084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-d13bccda-779b-42c7-9cee-829867c0b573 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:15.657084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:15.657084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:15.657084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:15.657084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:15.657084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:15.657084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:15.657084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:15.657084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:15.657084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:15.657084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:15.657084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:15.657084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:15.657084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:15.657084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:15.657084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:15.657926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:15.657926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:15.657926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:15.657926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:15.657926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:15.657926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:15.657926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:15.657926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:15.657926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:15.657926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:15.657926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:15.657926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:15.657926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:15.657926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:15.657926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:15.657926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:15.658794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:15.658794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:15.658794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:15.658794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:15.658794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:15.658794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:15.658794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:15.658794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:15.658794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:15.658794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:15.658794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:15.658794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:15.661629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 79/159] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:15.668128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee36d1-cbd3-4948-9073-4069be83fe24 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.791278 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:15.803508 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee36d1-cbd3-4948-9073-4069be83fe24 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:15.855395 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d708a69-09ff-485c-bc1b-6efd02bf9ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:15.856526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d708a69-09ff-485c-bc1b-6efd02bf9ca9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.857014 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d708a69-09ff-485c-bc1b-6efd02bf9ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.858030 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d708a69-09ff-485c-bc1b-6efd02bf9ca9 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.858981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d708a69-09ff-485c-bc1b-6efd02bf9ca9 None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:15.859504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d708a69-09ff-485c-bc1b-6efd02bf9ca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:15.861174 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d708a69-09ff-485c-bc1b-6efd02bf9ca9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:15.874939 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 81/160] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:15.881089 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bccda-779b-42c7-9cee-829867c0b573 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.897753 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb5825-b9d8-4165-b7e2-ced9afd01602 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.898356 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb5825-b9d8-4165-b7e2-ced9afd01602 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.898856 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb5825-b9d8-4165-b7e2-ced9afd01602 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.900576 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 82/161] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:15.906989 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c5ac2-5f3f-4d65-aca1-1a4fdb581d2d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.907591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c5ac2-5f3f-4d65-aca1-1a4fdb581d2d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.908102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c5ac2-5f3f-4d65-aca1-1a4fdb581d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.931757 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-633c5ac2-5f3f-4d65-aca1-1a4fdb581d2d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:15.933265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-633c5ac2-5f3f-4d65-aca1-1a4fdb581d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:16.041889 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bccda-779b-42c7-9cee-829867c0b573 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:16.053514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 83/162] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:16.110927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd5da2-18c7-480e-8774-2a0ac6f91302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:16.111595 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd5da2-18c7-480e-8774-2a0ac6f91302 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.111871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd5da2-18c7-480e-8774-2a0ac6f91302 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.112142 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd5da2-18c7-480e-8774-2a0ac6f91302 None admin] PATH_INFO: `/v3/users/9f12e9e4aad240d09cdbf43f68f765de` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.113399 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd5da2-18c7-480e-8774-2a0ac6f91302 None admin] RBAC: Authorizing `identity:get_user(user_id=9f12e9e4aad240d09cdbf43f68f765de)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:16.114868 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd5da2-18c7-480e-8774-2a0ac6f91302 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:16.116958 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 80/163] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:15 2022] GET /identity/v3/users/9f12e9e4aad240d09cdbf43f68f765de => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:16.122485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c5ac2-5f3f-4d65-aca1-1a4fdb581d2d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:16.176961 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026db91-1014-4b5b-b9c8-71400a550edd None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.177282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026db91-1014-4b5b-b9c8-71400a550edd None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.177611 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026db91-1014-4b5b-b9c8-71400a550edd None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.179058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 81/164] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:16.199809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfd599-7afc-41a0-afa6-3dec46a55ccc None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.200071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfd599-7afc-41a0-afa6-3dec46a55ccc None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.200279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfd599-7afc-41a0-afa6-3dec46a55ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.232352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-babfd599-7afc-41a0-afa6-3dec46a55ccc None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:16.233579 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-babfd599-7afc-41a0-afa6-3dec46a55ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:16.248574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:16.266443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c5ac2-5f3f-4d65-aca1-1a4fdb581d2d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:16.284629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:16.324286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20878854-dfac-4f2a-a341-74927c0b4c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:16.324929 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20878854-dfac-4f2a-a341-74927c0b4c04 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.325281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20878854-dfac-4f2a-a341-74927c0b4c04 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.325559 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20878854-dfac-4f2a-a341-74927c0b4c04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.357679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 82/165] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:16.442176 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20878854-dfac-4f2a-a341-74927c0b4c04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:16.443799 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20878854-dfac-4f2a-a341-74927c0b4c04 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:16.560539 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:16.580605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 84/166] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:16.586397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babfd599-7afc-41a0-afa6-3dec46a55ccc None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:16.718636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babfd599-7afc-41a0-afa6-3dec46a55ccc None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:16.761192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d9469-5302-42c6-9ad2-75693cf887f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:16.761751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d9469-5302-42c6-9ad2-75693cf887f0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.762023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d9469-5302-42c6-9ad2-75693cf887f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.762288 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d9469-5302-42c6-9ad2-75693cf887f0 None admin] PATH_INFO: `/v3/projects/4d4683865db24d41b5be961336407802` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.766602 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20878854-dfac-4f2a-a341-74927c0b4c04 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:16.767743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d9469-5302-42c6-9ad2-75693cf887f0 None admin] RBAC: Authorizing `identity:get_project(project_id=4d4683865db24d41b5be961336407802)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:16.769377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d9469-5302-42c6-9ad2-75693cf887f0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:16.771203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 83/167] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:16 2022] GET /identity/v3/projects/4d4683865db24d41b5be961336407802 => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:16.777231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d9469-5302-42c6-9ad2-75693cf887f0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:16.814323 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:16.910140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d9469-5302-42c6-9ad2-75693cf887f0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:16.960529 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145153d-7e11-45e5-8e8c-87e98445c45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:16.961589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145153d-7e11-45e5-8e8c-87e98445c45a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.962057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145153d-7e11-45e5-8e8c-87e98445c45a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.962516 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145153d-7e11-45e5-8e8c-87e98445c45a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.963492 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145153d-7e11-45e5-8e8c-87e98445c45a None admin] RBAC: Adding query filter params (role.id=cb722313950945e5b147e994ec678cd1, scope.project.id=4d4683865db24d41b5be961336407802, user.id=9f12e9e4aad240d09cdbf43f68f765de) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:16.964430 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145153d-7e11-45e5-8e8c-87e98445c45a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:16.966072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145153d-7e11-45e5-8e8c-87e98445c45a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:16.973661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 84/168] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:16 2022] GET /identity/v3/role_assignments?user.id=9f12e9e4aad240d09cdbf43f68f765de&scope.project.id=4d4683865db24d41b5be961336407802&role.id=cb722313950945e5b147e994ec678cd1 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:16.999959 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20878854-dfac-4f2a-a341-74927c0b4c04 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:17.005170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145153d-7e11-45e5-8e8c-87e98445c45a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:17.093121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb85f1-d893-4978-8dd4-98d76d280226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:17.095035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb85f1-d893-4978-8dd4-98d76d280226 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:17.095035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb85f1-d893-4978-8dd4-98d76d280226 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:17.095035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb85f1-d893-4978-8dd4-98d76d280226 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:17.150921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb85f1-d893-4978-8dd4-98d76d280226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:17.152434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-36fb85f1-d893-4978-8dd4-98d76d280226 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:17.152434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:17.152434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:17.152434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:17.152434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:17.152434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:17.152434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:17.152434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:17.152434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:17.152434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:17.152434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:17.152434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:17.152434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:17.152434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:17.152434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:17.152434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:17.153291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:17.153291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:17.153291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:17.153291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:17.153291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:17.153291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:17.153291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:17.153291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:17.153291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:17.153291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:17.153291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:17.153291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:17.153291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:17.153291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:17.153291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:17.153291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:17.154056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:17.154056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:17.154056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:17.154056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:17.154056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:17.154056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:17.154056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:17.154056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:17.154056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:17.154056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:17.154056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:17.154056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:17.154056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 85/169] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:17.160209 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:17.163944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb85f1-d893-4978-8dd4-98d76d280226 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:17.165584 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145153d-7e11-45e5-8e8c-87e98445c45a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:17.267670 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9482d4-80ac-4c7c-a69d-b4dc014d836a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:17.268416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9482d4-80ac-4c7c-a69d-b4dc014d836a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:17.268616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9482d4-80ac-4c7c-a69d-b4dc014d836a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:17.268899 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9482d4-80ac-4c7c-a69d-b4dc014d836a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:17.283284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:17.296349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9482d4-80ac-4c7c-a69d-b4dc014d836a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:17.301291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-4b9482d4-80ac-4c7c-a69d-b4dc014d836a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:17.301291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:17.301291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:17.301291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:17.301291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:17.301291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:17.301291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:17.301291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:17.301291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:17.301291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:17.301291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:17.301291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:17.301291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:17.301291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:17.301291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:17.301291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:17.302413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:17.302413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:17.302413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:17.302413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:17.302413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:17.302413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:17.302413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:17.302413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:17.302413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:17.302413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:17.302413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:17.302413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:17.302413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:17.302413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:17.302413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:17.302413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:17.303221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:17.303221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:17.303221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:17.303221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:17.303221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:17.303221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:17.303221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:17.303221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:17.303221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:17.303221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:17.303221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:17.303221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:17.303221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 85/170] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:17.312159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9482d4-80ac-4c7c-a69d-b4dc014d836a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:17.405452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb85f1-d893-4978-8dd4-98d76d280226 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:17.484657 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02133189-1895-410d-9229-143477549494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:17.485296 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02133189-1895-410d-9229-143477549494 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:17.485580 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02133189-1895-410d-9229-143477549494 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:17.485848 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02133189-1895-410d-9229-143477549494 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:17.486680 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02133189-1895-410d-9229-143477549494 None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:17.486884 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02133189-1895-410d-9229-143477549494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:17.494920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02133189-1895-410d-9229-143477549494 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:17.513692 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 86/171] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:17.522928 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02133189-1895-410d-9229-143477549494 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:17.549631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9482d4-80ac-4c7c-a69d-b4dc014d836a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:17.598796 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2dc67-8cd1-4720-b323-9daa2b162e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:17.600141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2dc67-8cd1-4720-b323-9daa2b162e96 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:17.600613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2dc67-8cd1-4720-b323-9daa2b162e96 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:17.601094 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2dc67-8cd1-4720-b323-9daa2b162e96 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:17.601950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2dc67-8cd1-4720-b323-9daa2b162e96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:17.602455 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2dc67-8cd1-4720-b323-9daa2b162e96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:17.604497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2dc67-8cd1-4720-b323-9daa2b162e96 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:17.619470 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 86/172] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:17.625175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2dc67-8cd1-4720-b323-9daa2b162e96 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:17.668119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:17.789339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02133189-1895-410d-9229-143477549494 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:17.797415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2dc67-8cd1-4720-b323-9daa2b162e96 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:17.803285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:17.847616 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed786e36-3af4-439e-bbe7-3c143a5137b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:17.848259 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed786e36-3af4-439e-bbe7-3c143a5137b7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:17.848540 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed786e36-3af4-439e-bbe7-3c143a5137b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:17.848814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed786e36-3af4-439e-bbe7-3c143a5137b7 None admin] PATH_INFO: `/v3/users/9f12e9e4aad240d09cdbf43f68f765de` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:17.849999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed786e36-3af4-439e-bbe7-3c143a5137b7 None admin] RBAC: Authorizing `identity:get_user(user_id=9f12e9e4aad240d09cdbf43f68f765de)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:17.850909 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fe6f5-740a-4fea-8d5e-66b206c0da79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:17.851492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed786e36-3af4-439e-bbe7-3c143a5137b7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:17.851597 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fe6f5-740a-4fea-8d5e-66b206c0da79 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:17.851808 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fe6f5-740a-4fea-8d5e-66b206c0da79 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:17.852077 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fe6f5-740a-4fea-8d5e-66b206c0da79 None admin] PATH_INFO: `/v3/users/c14eb0bf645e49f887a8d288d37d1491` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:17.853083 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 87/173] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:17 2022] GET /identity/v3/users/9f12e9e4aad240d09cdbf43f68f765de => generated 275 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:17.853515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9fe6f5-740a-4fea-8d5e-66b206c0da79 None admin] RBAC: Authorizing `identity:get_user(user_id=c14eb0bf645e49f887a8d288d37d1491)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:17.854197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d7640-b063-4a5c-b755-ca79a9cbdfeb None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:17.854475 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d7640-b063-4a5c-b755-ca79a9cbdfeb None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:17.854746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d7640-b063-4a5c-b755-ca79a9cbdfeb None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:17.854967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9fe6f5-740a-4fea-8d5e-66b206c0da79 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:17.855973 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 88/174] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:17.861876 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 87/175] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:17 2022] GET /identity/v3/users/c14eb0bf645e49f887a8d288d37d1491 => generated 247 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:17.867153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d7640-b063-4a5c-b755-ca79a9cbdfeb None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:17.867286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fe6f5-740a-4fea-8d5e-66b206c0da79 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:17.981220 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:18.006921 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fe6f5-740a-4fea-8d5e-66b206c0da79 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:18.037595 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d7640-b063-4a5c-b755-ca79a9cbdfeb None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:18.081770 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd7cb7-ae4c-40fd-9e69-375f57da2af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:18.082755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd7cb7-ae4c-40fd-9e69-375f57da2af8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.083248 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd7cb7-ae4c-40fd-9e69-375f57da2af8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.083800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd7cb7-ae4c-40fd-9e69-375f57da2af8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.102390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d3f26-2e39-4231-98f1-d3bf7f744958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:18.103412 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d3f26-2e39-4231-98f1-d3bf7f744958 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.105324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d3f26-2e39-4231-98f1-d3bf7f744958 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.105810 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d3f26-2e39-4231-98f1-d3bf7f744958 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.185028 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:18.204846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd7cb7-ae4c-40fd-9e69-375f57da2af8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:18.206815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd7cb7-ae4c-40fd-9e69-375f57da2af8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:18.222755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d3f26-2e39-4231-98f1-d3bf7f744958 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:18.225268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d3f26-2e39-4231-98f1-d3bf7f744958 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:18.302762 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:18.315745 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 88/176] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:18.316895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ad172-227e-41e6-a374-83296ee21e87 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.317177 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ad172-227e-41e6-a374-83296ee21e87 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.320690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ad172-227e-41e6-a374-83296ee21e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.338180 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-954ad172-227e-41e6-a374-83296ee21e87 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:18.340114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-954ad172-227e-41e6-a374-83296ee21e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:18.344523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 89/177] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:18.344523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab9b31-65ab-415b-9f23-07f8c7d8f593 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.344523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab9b31-65ab-415b-9f23-07f8c7d8f593 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.344523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab9b31-65ab-415b-9f23-07f8c7d8f593 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.344523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 90/178] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:18.352644 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf5533-a0b3-4005-87cc-56e73c8a309d None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.352925 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf5533-a0b3-4005-87cc-56e73c8a309d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.353199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf5533-a0b3-4005-87cc-56e73c8a309d None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.354434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 91/179] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:18.354434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf5533-a0b3-4005-87cc-56e73c8a309d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:18.414959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 89/180] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:18.416151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c406b6-78ca-4da9-ae0a-f62fd271d4cb None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.416441 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c406b6-78ca-4da9-ae0a-f62fd271d4cb None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.416719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c406b6-78ca-4da9-ae0a-f62fd271d4cb None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.422564 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 90/181] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:18.422564 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c406b6-78ca-4da9-ae0a-f62fd271d4cb None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:18.431744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:18.464146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf5533-a0b3-4005-87cc-56e73c8a309d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:18.507554 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f52bb2-7ccb-48df-8ce2-2184321097bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:18.508181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f52bb2-7ccb-48df-8ce2-2184321097bb None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.508749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f52bb2-7ccb-48df-8ce2-2184321097bb None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.509115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f52bb2-7ccb-48df-8ce2-2184321097bb None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.510118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f52bb2-7ccb-48df-8ce2-2184321097bb None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:18.511588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f52bb2-7ccb-48df-8ce2-2184321097bb None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:18.513198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 92/182] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:18 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:18.514308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6dfbc-d66e-4beb-b15e-c7ab0f09e86d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.514580 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6dfbc-d66e-4beb-b15e-c7ab0f09e86d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.514846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6dfbc-d66e-4beb-b15e-c7ab0f09e86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.532984 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-73e6dfbc-d66e-4beb-b15e-c7ab0f09e86d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:18.534145 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-73e6dfbc-d66e-4beb-b15e-c7ab0f09e86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:18.539161 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c406b6-78ca-4da9-ae0a-f62fd271d4cb None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:18.558503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:18.574502 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b349ab-9fda-4ff4-a4e7-ae917f43c348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:18.575154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b349ab-9fda-4ff4-a4e7-ae917f43c348 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.575434 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b349ab-9fda-4ff4-a4e7-ae917f43c348 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.575708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b349ab-9fda-4ff4-a4e7-ae917f43c348 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.577101 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b349ab-9fda-4ff4-a4e7-ae917f43c348 None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:18.578549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b349ab-9fda-4ff4-a4e7-ae917f43c348 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:18.580205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 91/183] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:18 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:18.581298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54ad30-596a-45bc-8b86-615bcfaca4f5 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.581578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54ad30-596a-45bc-8b86-615bcfaca4f5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.581847 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54ad30-596a-45bc-8b86-615bcfaca4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.600834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-2d54ad30-596a-45bc-8b86-615bcfaca4f5 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:18.602107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2d54ad30-596a-45bc-8b86-615bcfaca4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:18.612286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 93/184] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:18.613443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e028a-b62c-4137-9593-f0f50f9a822b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.613723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e028a-b62c-4137-9593-f0f50f9a822b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.613997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e028a-b62c-4137-9593-f0f50f9a822b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.637480 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-925e028a-b62c-4137-9593-f0f50f9a822b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:18.638677 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-925e028a-b62c-4137-9593-f0f50f9a822b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:18.698864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 92/185] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:18.699766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54ad30-596a-45bc-8b86-615bcfaca4f5 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:18.723405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:18.728475 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 94/186] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:18.729412 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e028a-b62c-4137-9593-f0f50f9a822b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:18.832405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54ad30-596a-45bc-8b86-615bcfaca4f5 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:18.851435 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e028a-b62c-4137-9593-f0f50f9a822b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:18.858486 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:18.885872 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4eed3-65fe-4dbe-8dde-7853f78053d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:18.886999 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4eed3-65fe-4dbe-8dde-7853f78053d1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.887554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4eed3-65fe-4dbe-8dde-7853f78053d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.888006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4eed3-65fe-4dbe-8dde-7853f78053d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.889115 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4eed3-65fe-4dbe-8dde-7853f78053d1 None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72, user.id=c14eb0bf645e49f887a8d288d37d1491) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:18.889364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea088b-68f0-4b75-8b1e-f7a05a868ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:18.889828 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4eed3-65fe-4dbe-8dde-7853f78053d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:18.890084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea088b-68f0-4b75-8b1e-f7a05a868ce9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.890427 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea088b-68f0-4b75-8b1e-f7a05a868ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.890774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea088b-68f0-4b75-8b1e-f7a05a868ce9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.891587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea088b-68f0-4b75-8b1e-f7a05a868ce9 None admin] RBAC: Adding query filter params (role.id=cb722313950945e5b147e994ec678cd1, scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72, user.id=9f12e9e4aad240d09cdbf43f68f765de) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:18.891955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea088b-68f0-4b75-8b1e-f7a05a868ce9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:18.892482 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4eed3-65fe-4dbe-8dde-7853f78053d1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:18.893511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea088b-68f0-4b75-8b1e-f7a05a868ce9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:18.901093 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 95/187] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:18 2022] GET /identity/v3/role_assignments?user.id=9f12e9e4aad240d09cdbf43f68f765de&scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72&role.id=cb722313950945e5b147e994ec678cd1 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:18.901541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 93/188] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:18 2022] GET /identity/v3/role_assignments?user.id=c14eb0bf645e49f887a8d288d37d1491&scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:18.910952 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4eed3-65fe-4dbe-8dde-7853f78053d1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:18.911151 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea088b-68f0-4b75-8b1e-f7a05a868ce9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:19.056805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:19.080086 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4eed3-65fe-4dbe-8dde-7853f78053d1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:19.130149 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea088b-68f0-4b75-8b1e-f7a05a868ce9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:19.131346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f74be-eccf-482e-a276-ba1140787bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:19.132526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f74be-eccf-482e-a276-ba1140787bbc None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.134096 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f74be-eccf-482e-a276-ba1140787bbc None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.134602 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f74be-eccf-482e-a276-ba1140787bbc None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.136164 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f74be-eccf-482e-a276-ba1140787bbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:19.137871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f74be-eccf-482e-a276-ba1140787bbc None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:19.140621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 94/189] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:19.141397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f74be-eccf-482e-a276-ba1140787bbc None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:19.172736 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e67c29-9f76-4574-8177-5bffc07cc272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:19.173395 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e67c29-9f76-4574-8177-5bffc07cc272 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.173741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e67c29-9f76-4574-8177-5bffc07cc272 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.174022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e67c29-9f76-4574-8177-5bffc07cc272 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.188031 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e67c29-9f76-4574-8177-5bffc07cc272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:19.193744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-c5e67c29-9f76-4574-8177-5bffc07cc272 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:19.193744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:19.193744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:19.193744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:19.193744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:19.193744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:19.193744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:19.193744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:19.193744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:19.193744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:19.193744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:19.193744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:19.193744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:19.193744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:19.193744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:19.193744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:19.194703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:19.194703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:19.194703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:19.194703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:19.194703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:19.194703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:19.194703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:19.194703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:19.194703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:19.194703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:19.194703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:19.194703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:19.194703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:19.194703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:19.194703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:19.194703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:19.195521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:19.195521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:19.195521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:19.195521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:19.195521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:19.195521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:19.195521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:19.195521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:19.195521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:19.195521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:19.195521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:19.195521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:19.195521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 96/190] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:19.196209 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e67c29-9f76-4574-8177-5bffc07cc272 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:19.221415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:19.282639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f74be-eccf-482e-a276-ba1140787bbc None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:19.323176 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e67c29-9f76-4574-8177-5bffc07cc272 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:19.324235 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c0587-b882-4024-a7f9-1fb1646a39e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:19.324873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c0587-b882-4024-a7f9-1fb1646a39e6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.325148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c0587-b882-4024-a7f9-1fb1646a39e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.325420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c0587-b882-4024-a7f9-1fb1646a39e6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.336560 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c0587-b882-4024-a7f9-1fb1646a39e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:19.341785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-9c2c0587-b882-4024-a7f9-1fb1646a39e6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:19.341785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:19.341785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:19.341785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:19.341785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:19.341785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:19.341785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:19.341785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:19.341785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:19.341785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:19.341785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:19.341785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:19.341785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:19.341785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:19.341785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:19.341785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:19.343350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:19.343350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:19.343350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:19.343350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:19.343350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:19.343350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:19.343350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:19.343350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:19.343350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:19.343350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:19.343350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:19.343350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:19.343350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:19.343350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:19.343350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:19.343350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:19.344249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:19.344249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:19.344249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:19.344249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:19.344249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:19.344249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:19.344249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:19.344249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:19.344249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:19.344249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:19.344249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:19.344249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:19.344249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 95/191] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:19.347682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c0587-b882-4024-a7f9-1fb1646a39e6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:19.349023 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:19.359631 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd429edf-25ce-4692-aa95-b969fa8ea824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:19.360273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd429edf-25ce-4692-aa95-b969fa8ea824 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.360555 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd429edf-25ce-4692-aa95-b969fa8ea824 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.360830 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd429edf-25ce-4692-aa95-b969fa8ea824 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.371458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd429edf-25ce-4692-aa95-b969fa8ea824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:19.376794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-cd429edf-25ce-4692-aa95-b969fa8ea824 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:19.376794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:19.376794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:19.376794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:19.376794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:19.376794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:19.376794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:19.376794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:19.376794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:19.376794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:19.376794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:19.376794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:19.376794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:19.376794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:19.376794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:19.376794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:19.377649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:19.377649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:19.377649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:19.377649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:19.377649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:19.377649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:19.377649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:19.377649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:19.377649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:19.377649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:19.377649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:19.377649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:19.377649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:19.377649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:19.377649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:19.377649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:19.378659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:19.378659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:19.378659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:19.378659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:19.378659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:19.378659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:19.378659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:19.378659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:19.378659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:19.378659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:19.378659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:19.378659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:19.378659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 97/192] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:19.378659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd429edf-25ce-4692-aa95-b969fa8ea824 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:19.450660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c0587-b882-4024-a7f9-1fb1646a39e6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:19.460027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:19.486809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117e7b8-83a3-422b-a4e1-a723651777d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:19.487434 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117e7b8-83a3-422b-a4e1-a723651777d9 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.487707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117e7b8-83a3-422b-a4e1-a723651777d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.487973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117e7b8-83a3-422b-a4e1-a723651777d9 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.488653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117e7b8-83a3-422b-a4e1-a723651777d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:19.490094 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117e7b8-83a3-422b-a4e1-a723651777d9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:19.519579 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd429edf-25ce-4692-aa95-b969fa8ea824 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:19.553309 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee83ad-008f-4697-adc8-47286848848f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:19.553906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee83ad-008f-4697-adc8-47286848848f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.554184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee83ad-008f-4697-adc8-47286848848f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.554458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee83ad-008f-4697-adc8-47286848848f None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.555147 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee83ad-008f-4697-adc8-47286848848f None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:19.555445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee83ad-008f-4697-adc8-47286848848f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:19.556823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee83ad-008f-4697-adc8-47286848848f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:19.564863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 98/193] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:19.565687 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee83ad-008f-4697-adc8-47286848848f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:19.602724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 96/194] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:37:19 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:19.602724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117e7b8-83a3-422b-a4e1-a723651777d9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:19.615212 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:19.682273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee83ad-008f-4697-adc8-47286848848f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:19.722487 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425917f-fdb6-42ad-a3ee-763e0ffe3e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:19.723052 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425917f-fdb6-42ad-a3ee-763e0ffe3e5a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.723333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425917f-fdb6-42ad-a3ee-763e0ffe3e5a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.723605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425917f-fdb6-42ad-a3ee-763e0ffe3e5a None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.724168 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117e7b8-83a3-422b-a4e1-a723651777d9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:19.724279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425917f-fdb6-42ad-a3ee-763e0ffe3e5a None admin] RBAC: Adding query filter params (name=reader) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:19.724563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425917f-fdb6-42ad-a3ee-763e0ffe3e5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:19.726113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425917f-fdb6-42ad-a3ee-763e0ffe3e5a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:19.736155 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 99/195] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:19.736927 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425917f-fdb6-42ad-a3ee-763e0ffe3e5a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:19.767096 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e9e6a-b3be-4288-a32a-5d7a98d4e617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:19.767716 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e9e6a-b3be-4288-a32a-5d7a98d4e617 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.767992 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e9e6a-b3be-4288-a32a-5d7a98d4e617 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.768267 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e9e6a-b3be-4288-a32a-5d7a98d4e617 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.768927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e9e6a-b3be-4288-a32a-5d7a98d4e617 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:19.769229 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e9e6a-b3be-4288-a32a-5d7a98d4e617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:19.770625 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e9e6a-b3be-4288-a32a-5d7a98d4e617 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:19.779525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 97/196] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:19.780731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e9e6a-b3be-4288-a32a-5d7a98d4e617 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:19.813566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:19.860264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425917f-fdb6-42ad-a3ee-763e0ffe3e5a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:19.872401 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:19.901785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e9e6a-b3be-4288-a32a-5d7a98d4e617 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:19.902887 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8c927-2653-4224-a47d-f9f8350fa013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:19.903546 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8c927-2653-4224-a47d-f9f8350fa013 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.903822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8c927-2653-4224-a47d-f9f8350fa013 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.904096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8c927-2653-4224-a47d-f9f8350fa013 None admin] PATH_INFO: `/v3/users/32db92767a194b7196757e0f4d802bd6` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.921095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8c927-2653-4224-a47d-f9f8350fa013 None admin] RBAC: Authorizing `identity:get_user(user_id=32db92767a194b7196757e0f4d802bd6)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:19.923266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8c927-2653-4224-a47d-f9f8350fa013 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:19.925699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 100/197] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:19 2022] GET /identity/v3/users/32db92767a194b7196757e0f4d802bd6 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:19.926325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8c927-2653-4224-a47d-f9f8350fa013 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:19.949794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe05e82-4833-4cc9-bda8-519794636bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:19.950668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe05e82-4833-4cc9-bda8-519794636bea None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.951494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe05e82-4833-4cc9-bda8-519794636bea None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.951930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe05e82-4833-4cc9-bda8-519794636bea None admin] PATH_INFO: `/v3/users/cf0367416938434197b41766ae665b2b` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.953301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe05e82-4833-4cc9-bda8-519794636bea None admin] RBAC: Authorizing `identity:get_user(user_id=cf0367416938434197b41766ae665b2b)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:19.955052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe05e82-4833-4cc9-bda8-519794636bea None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:19.957095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 98/198] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:19 2022] GET /identity/v3/users/cf0367416938434197b41766ae665b2b => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:19.958032 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe05e82-4833-4cc9-bda8-519794636bea None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.059843 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:20.065597 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8c927-2653-4224-a47d-f9f8350fa013 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.068146 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe05e82-4833-4cc9-bda8-519794636bea None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.104368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daa45d-7a22-4063-b7bb-499cde34f5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.105388 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daa45d-7a22-4063-b7bb-499cde34f5df None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.105863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daa45d-7a22-4063-b7bb-499cde34f5df None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.106420 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daa45d-7a22-4063-b7bb-499cde34f5df None admin] PATH_INFO: `/v3/users/32db92767a194b7196757e0f4d802bd6` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.107949 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02daa45d-7a22-4063-b7bb-499cde34f5df None admin] RBAC: Authorizing `identity:get_user(user_id=32db92767a194b7196757e0f4d802bd6)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.109848 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02daa45d-7a22-4063-b7bb-499cde34f5df None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:20.112018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc01fb-e23a-4e3e-906b-5aac8a142cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.112281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 101/199] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:19 2022] GET /identity/v3/users/32db92767a194b7196757e0f4d802bd6 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:20.112657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc01fb-e23a-4e3e-906b-5aac8a142cdd None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.113000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc01fb-e23a-4e3e-906b-5aac8a142cdd None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.113273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc01fb-e23a-4e3e-906b-5aac8a142cdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.113988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daa45d-7a22-4063-b7bb-499cde34f5df None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.159340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:20.192951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc01fb-e23a-4e3e-906b-5aac8a142cdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.194821 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc01fb-e23a-4e3e-906b-5aac8a142cdd None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:20.241880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daa45d-7a22-4063-b7bb-499cde34f5df None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.278368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03178e10-0432-4212-8d81-ac6b9a143cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.279390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03178e10-0432-4212-8d81-ac6b9a143cfb None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.279896 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03178e10-0432-4212-8d81-ac6b9a143cfb None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.280364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03178e10-0432-4212-8d81-ac6b9a143cfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.313621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:20.324130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 99/200] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:20.329848 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc01fb-e23a-4e3e-906b-5aac8a142cdd None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.371026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03178e10-0432-4212-8d81-ac6b9a143cfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.372866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03178e10-0432-4212-8d81-ac6b9a143cfb None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:20.410407 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:20.476538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc01fb-e23a-4e3e-906b-5aac8a142cdd None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.497756 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 102/201] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:20.498988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03178e10-0432-4212-8d81-ac6b9a143cfb None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.520977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d5e2f-56c2-4913-b9df-a9b1bf6c8bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.521895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d5e2f-56c2-4913-b9df-a9b1bf6c8bf0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.522316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d5e2f-56c2-4913-b9df-a9b1bf6c8bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.522722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d5e2f-56c2-4913-b9df-a9b1bf6c8bf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.616047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:20.619473 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03178e10-0432-4212-8d81-ac6b9a143cfb None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.625250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d5e2f-56c2-4913-b9df-a9b1bf6c8bf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.626887 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d5e2f-56c2-4913-b9df-a9b1bf6c8bf0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:20.657208 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc6570-5e72-48b2-875c-44220ecb7f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.658245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc6570-5e72-48b2-875c-44220ecb7f70 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.658716 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc6570-5e72-48b2-875c-44220ecb7f70 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.659200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc6570-5e72-48b2-875c-44220ecb7f70 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.661527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc6570-5e72-48b2-875c-44220ecb7f70 None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.663225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc6570-5e72-48b2-875c-44220ecb7f70 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:20.665954 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 103/202] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:20.666587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc6570-5e72-48b2-875c-44220ecb7f70 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.673352 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:20.763482 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 100/203] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:20.764165 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d5e2f-56c2-4913-b9df-a9b1bf6c8bf0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.796198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc6570-5e72-48b2-875c-44220ecb7f70 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.835441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe660e-15a3-4b6b-b0e4-9bc7569d8ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.836068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe660e-15a3-4b6b-b0e4-9bc7569d8ef8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.836341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe660e-15a3-4b6b-b0e4-9bc7569d8ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.836610 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe660e-15a3-4b6b-b0e4-9bc7569d8ef8 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.837792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe660e-15a3-4b6b-b0e4-9bc7569d8ef8 None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.839493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe660e-15a3-4b6b-b0e4-9bc7569d8ef8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:20.841878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 104/204] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:20.843488 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe660e-15a3-4b6b-b0e4-9bc7569d8ef8 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.862192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:20.868963 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d5e2f-56c2-4913-b9df-a9b1bf6c8bf0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.910637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b106056-5cec-4344-a090-a9e67978751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.912642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b106056-5cec-4344-a090-a9e67978751e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.913726 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b106056-5cec-4344-a090-a9e67978751e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.914205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b106056-5cec-4344-a090-a9e67978751e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.915455 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b106056-5cec-4344-a090-a9e67978751e None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=8c60d15f044842a0bc451a2e001075f0, user.id=32db92767a194b7196757e0f4d802bd6) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:20.916240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b106056-5cec-4344-a090-a9e67978751e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.918597 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b106056-5cec-4344-a090-a9e67978751e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:20.935390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 101/205] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/role_assignments?user.id=32db92767a194b7196757e0f4d802bd6&scope.project.id=8c60d15f044842a0bc451a2e001075f0&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:20.935390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b106056-5cec-4344-a090-a9e67978751e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.939462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:21.001728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe660e-15a3-4b6b-b0e4-9bc7569d8ef8 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:21.070647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db3581-6f32-4776-bc29-7a6bbfc7569d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:21.071373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db3581-6f32-4776-bc29-7a6bbfc7569d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:21.071647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db3581-6f32-4776-bc29-7a6bbfc7569d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:21.071970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db3581-6f32-4776-bc29-7a6bbfc7569d None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:21.073903 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db3581-6f32-4776-bc29-7a6bbfc7569d None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:21.075402 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db3581-6f32-4776-bc29-7a6bbfc7569d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:21.082456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 105/206] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:21.087974 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db3581-6f32-4776-bc29-7a6bbfc7569d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:21.136331 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b106056-5cec-4344-a090-a9e67978751e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:21.211607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf4e6b-1d59-4f68-841e-d46d93de757e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:21.212224 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf4e6b-1d59-4f68-841e-d46d93de757e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:21.212501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf4e6b-1d59-4f68-841e-d46d93de757e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:21.212769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf4e6b-1d59-4f68-841e-d46d93de757e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:21.213602 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf4e6b-1d59-4f68-841e-d46d93de757e None admin] RBAC: Adding query filter params (role.id=d3fb1e3ec057480992710ad61b6cd1ed, scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72, user.id=cf0367416938434197b41766ae665b2b) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:21.213912 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf4e6b-1d59-4f68-841e-d46d93de757e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:21.215497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf4e6b-1d59-4f68-841e-d46d93de757e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:21.229641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 102/207] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/role_assignments?user.id=cf0367416938434197b41766ae665b2b&scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72&role.id=d3fb1e3ec057480992710ad61b6cd1ed => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:21.242538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db3581-6f32-4776-bc29-7a6bbfc7569d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:21.280850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edb8e3-4c89-453f-852e-51919d1dbe4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:21.281913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edb8e3-4c89-453f-852e-51919d1dbe4a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:21.282442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edb8e3-4c89-453f-852e-51919d1dbe4a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:21.282937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edb8e3-4c89-453f-852e-51919d1dbe4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:21.283948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edb8e3-4c89-453f-852e-51919d1dbe4a None admin] RBAC: Adding query filter params (role.id=1a62bc555eb345fc8679cfa677069fd7, scope.project.id=8c60d15f044842a0bc451a2e001075f0, user.id=32db92767a194b7196757e0f4d802bd6) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:21.284510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edb8e3-4c89-453f-852e-51919d1dbe4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:21.286225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edb8e3-4c89-453f-852e-51919d1dbe4a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:21.293334 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 106/208] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:21 2022] GET /identity/v3/role_assignments?user.id=32db92767a194b7196757e0f4d802bd6&scope.project.id=8c60d15f044842a0bc451a2e001075f0&role.id=1a62bc555eb345fc8679cfa677069fd7 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:22.047333 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3588842-b89d-45fa-bdfd-ac5c9d61d936 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:22.047878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3588842-b89d-45fa-bdfd-ac5c9d61d936 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:22.047878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3588842-b89d-45fa-bdfd-ac5c9d61d936 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:22.049387 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 103/209] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:22.054725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94f513-6e77-4859-87a4-2197a07b8d55 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:22.055158 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94f513-6e77-4859-87a4-2197a07b8d55 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:22.055330 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94f513-6e77-4859-87a4-2197a07b8d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:22.062125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:22.075674 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7e94f513-6e77-4859-87a4-2197a07b8d55 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:22.077487 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7e94f513-6e77-4859-87a4-2197a07b8d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:22.114490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac3287-3024-4fbe-94d0-ea549c728d09 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:22.114797 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac3287-3024-4fbe-94d0-ea549c728d09 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:22.115073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac3287-3024-4fbe-94d0-ea549c728d09 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:22.116646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 104/210] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:22.121628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4f5f9-45f5-4f0d-8ad4-62f5a7cf5e13 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:22.121907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4f5f9-45f5-4f0d-8ad4-62f5a7cf5e13 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:22.122182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4f5f9-45f5-4f0d-8ad4-62f5a7cf5e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:22.140049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-21b4f5f9-45f5-4f0d-8ad4-62f5a7cf5e13 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:22.141232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-21b4f5f9-45f5-4f0d-8ad4-62f5a7cf5e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:22.156549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 107/211] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:22.164290 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:22.218129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 105/212] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:22.436719 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94f513-6e77-4859-87a4-2197a07b8d55 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:22.525671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4f5f9-45f5-4f0d-8ad4-62f5a7cf5e13 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:22.550819 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94f513-6e77-4859-87a4-2197a07b8d55 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:22.585165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:22.590427 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab482189-d3c7-4529-9b10-b83059c1c50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:22.591091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab482189-d3c7-4529-9b10-b83059c1c50a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:22.591372 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab482189-d3c7-4529-9b10-b83059c1c50a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:22.591643 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab482189-d3c7-4529-9b10-b83059c1c50a None admin] PATH_INFO: `/v3/domains/default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:22.594024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab482189-d3c7-4529-9b10-b83059c1c50a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:22.595526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab482189-d3c7-4529-9b10-b83059c1c50a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:22.598969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 108/213] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:22.604172 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab482189-d3c7-4529-9b10-b83059c1c50a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:22.635687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4f5f9-45f5-4f0d-8ad4-62f5a7cf5e13 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:22.675158 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a0c67-95fa-4331-8a97-c154ac6b64e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:22.676190 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a0c67-95fa-4331-8a97-c154ac6b64e5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:22.676689 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a0c67-95fa-4331-8a97-c154ac6b64e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:22.677249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a0c67-95fa-4331-8a97-c154ac6b64e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:22.692064 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a0c67-95fa-4331-8a97-c154ac6b64e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:22.698197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-2e3a0c67-95fa-4331-8a97-c154ac6b64e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:22.698197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:22.698197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:22.698197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:22.698197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:22.698197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:22.698197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:22.698197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:22.698197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:22.698197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:22.698197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:22.698197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:22.698197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:22.698197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:22.698197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:22.698197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:22.699160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:22.699160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:22.699160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:22.699160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:22.699160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:22.699160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:22.699160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:22.699160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:22.699160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:22.699160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:22.699160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:22.699160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:22.699160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:22.699160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:22.699160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:22.699160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:22.699923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:22.699923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:22.699923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:22.699923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:22.699923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:22.699923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:22.699923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:22.699923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:22.699923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:22.699923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:22.699923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:22.699923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:22.699923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 106/214] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:22.707953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a0c67-95fa-4331-8a97-c154ac6b64e5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:22.725231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:22.736367 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab482189-d3c7-4529-9b10-b83059c1c50a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:22.784095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5d2cc-275d-41a9-9811-103f1b604493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:22.784701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5d2cc-275d-41a9-9811-103f1b604493 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:22.785009 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5d2cc-275d-41a9-9811-103f1b604493 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:22.785835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5d2cc-275d-41a9-9811-103f1b604493 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:22.786505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5d2cc-275d-41a9-9811-103f1b604493 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:22.788275 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5d2cc-275d-41a9-9811-103f1b604493 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:22.823849 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a0c67-95fa-4331-8a97-c154ac6b64e5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:22.861481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aeaf58-b552-46b5-b9d4-4034b2cc1e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:22.862084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aeaf58-b552-46b5-b9d4-4034b2cc1e36 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:22.862361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aeaf58-b552-46b5-b9d4-4034b2cc1e36 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:22.862630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aeaf58-b552-46b5-b9d4-4034b2cc1e36 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:22.863331 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aeaf58-b552-46b5-b9d4-4034b2cc1e36 None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:22.863630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aeaf58-b552-46b5-b9d4-4034b2cc1e36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:22.865007 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aeaf58-b552-46b5-b9d4-4034b2cc1e36 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:22.878294 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 107/215] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:22.879286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aeaf58-b552-46b5-b9d4-4034b2cc1e36 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:22.895292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 109/216] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:37:22 2022] POST /identity/v3/users => generated 297 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:37:22.976416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:23.006964 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aeaf58-b552-46b5-b9d4-4034b2cc1e36 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:23.047161 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58587e6-7ccd-4824-9901-79dbdad38b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:23.048179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58587e6-7ccd-4824-9901-79dbdad38b97 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:23.049235 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58587e6-7ccd-4824-9901-79dbdad38b97 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:23.049733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58587e6-7ccd-4824-9901-79dbdad38b97 None admin] PATH_INFO: `/v3/users/fb74804ffc39429e9a10ef19845a532d` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:23.067601 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58587e6-7ccd-4824-9901-79dbdad38b97 None admin] RBAC: Authorizing `identity:get_user(user_id=fb74804ffc39429e9a10ef19845a532d)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:23.069584 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58587e6-7ccd-4824-9901-79dbdad38b97 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:23.071620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 108/217] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:22 2022] GET /identity/v3/users/fb74804ffc39429e9a10ef19845a532d => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:23.079423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5d2cc-275d-41a9-9811-103f1b604493 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:23.148473 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:23.198460 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5d2cc-275d-41a9-9811-103f1b604493 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:23.236905 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66e3b9-66ce-4435-b355-cd244e74272e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:23.237445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66e3b9-66ce-4435-b355-cd244e74272e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:23.237721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66e3b9-66ce-4435-b355-cd244e74272e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:23.237992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66e3b9-66ce-4435-b355-cd244e74272e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:23.258713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5892c-6678-4bba-83cb-a315d6ece664 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:23.259123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5892c-6678-4bba-83cb-a315d6ece664 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:23.259402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5892c-6678-4bba-83cb-a315d6ece664 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:23.260739 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 109/218] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:23.267306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024597e6-b5a3-4d95-9da6-72490b18c130 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:23.267586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024597e6-b5a3-4d95-9da6-72490b18c130 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:23.267859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024597e6-b5a3-4d95-9da6-72490b18c130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:23.294994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-024597e6-b5a3-4d95-9da6-72490b18c130 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:23.296744 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-024597e6-b5a3-4d95-9da6-72490b18c130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:23.323944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f66e3b9-66ce-4435-b355-cd244e74272e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:23.325433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f66e3b9-66ce-4435-b355-cd244e74272e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:23.385728 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 110/219] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:23.454699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:23.522195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 110/220] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:23.523734 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024597e6-b5a3-4d95-9da6-72490b18c130 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:23.636586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:23.685559 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024597e6-b5a3-4d95-9da6-72490b18c130 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:23.720191 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8325cd3-74ba-4695-a9f7-18b1c23fda33 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:23.720563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8325cd3-74ba-4695-a9f7-18b1c23fda33 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:23.720915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8325cd3-74ba-4695-a9f7-18b1c23fda33 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:23.723212 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 111/221] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:23.730101 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291ac1f-5401-4c5d-8924-6a204f882794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:23.730738 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291ac1f-5401-4c5d-8924-6a204f882794 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:23.731059 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291ac1f-5401-4c5d-8924-6a204f882794 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:23.731332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291ac1f-5401-4c5d-8924-6a204f882794 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:23.733057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291ac1f-5401-4c5d-8924-6a204f882794 None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:23.734543 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb891f6-c3b2-41f9-bfa3-380c1090cb43 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:23.734832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb891f6-c3b2-41f9-bfa3-380c1090cb43 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:23.735019 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6291ac1f-5401-4c5d-8924-6a204f882794 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:23.735128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb891f6-c3b2-41f9-bfa3-380c1090cb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:23.736816 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 111/222] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:23 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:23.742558 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291ac1f-5401-4c5d-8924-6a204f882794 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:23.759652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-feb891f6-c3b2-41f9-bfa3-380c1090cb43 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:23.760846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-feb891f6-c3b2-41f9-bfa3-380c1090cb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:23.853870 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 112/223] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:23.855872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb891f6-c3b2-41f9-bfa3-380c1090cb43 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:23.859094 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291ac1f-5401-4c5d-8924-6a204f882794 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:23.890525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:23.897057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b2faa-911c-4035-9a74-8d9bf9befbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:23.897541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b2faa-911c-4035-9a74-8d9bf9befbc5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:23.897819 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b2faa-911c-4035-9a74-8d9bf9befbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:23.898091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b2faa-911c-4035-9a74-8d9bf9befbc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:23.898815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b2faa-911c-4035-9a74-8d9bf9befbc5 None admin] RBAC: Adding query filter params (role.id=cb722313950945e5b147e994ec678cd1, scope.project.id=8c60d15f044842a0bc451a2e001075f0, user.id=fb74804ffc39429e9a10ef19845a532d) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:23.899157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b2faa-911c-4035-9a74-8d9bf9befbc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:23.900651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b2faa-911c-4035-9a74-8d9bf9befbc5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:23.909062 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 112/224] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:23 2022] GET /identity/v3/role_assignments?user.id=fb74804ffc39429e9a10ef19845a532d&scope.project.id=8c60d15f044842a0bc451a2e001075f0&role.id=cb722313950945e5b147e994ec678cd1 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:23.998920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:24.006615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb891f6-c3b2-41f9-bfa3-380c1090cb43 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:24.059005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca2529-aa2d-488a-b274-9a8457160b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:24.059973 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca2529-aa2d-488a-b274-9a8457160b06 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.060441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca2529-aa2d-488a-b274-9a8457160b06 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.060958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca2529-aa2d-488a-b274-9a8457160b06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.078214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca2529-aa2d-488a-b274-9a8457160b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:24.083457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-0eca2529-aa2d-488a-b274-9a8457160b06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:24.083457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:24.083457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:24.083457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:24.083457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:24.083457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:24.083457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:24.083457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:24.083457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:24.083457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:24.083457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:24.083457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:24.083457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:24.083457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:24.083457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:24.083457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:24.085175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:24.085175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:24.085175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:24.085175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:24.085175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:24.085175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:24.085175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:24.085175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:24.085175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:24.085175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:24.085175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:24.085175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:24.085175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:24.085175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:24.085175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:24.085175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:24.085987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:24.085987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:24.085987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:24.085987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:24.085987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:24.085987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:24.085987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:24.085987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:24.085987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:24.085987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:24.085987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:24.085987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:24.088019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 113/225] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:24.094939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b2faa-911c-4035-9a74-8d9bf9befbc5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:24.162850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca2529-aa2d-488a-b274-9a8457160b06 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:24.255603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b2faa-911c-4035-9a74-8d9bf9befbc5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:24.322916 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54a8b8-2ac7-4d2a-b27c-df1abdc5957f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:24.325412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54a8b8-2ac7-4d2a-b27c-df1abdc5957f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.327412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54a8b8-2ac7-4d2a-b27c-df1abdc5957f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.327880 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54a8b8-2ac7-4d2a-b27c-df1abdc5957f None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.328729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54a8b8-2ac7-4d2a-b27c-df1abdc5957f None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:24.330048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54a8b8-2ac7-4d2a-b27c-df1abdc5957f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:24.331654 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54a8b8-2ac7-4d2a-b27c-df1abdc5957f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:24.350922 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 113/226] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:24.356765 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54a8b8-2ac7-4d2a-b27c-df1abdc5957f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:24.365444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca2529-aa2d-488a-b274-9a8457160b06 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:24.444095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e8c53-8e00-41ce-9727-5d906627c1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:24.447215 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e8c53-8e00-41ce-9727-5d906627c1c6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.447698 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e8c53-8e00-41ce-9727-5d906627c1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.448141 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e8c53-8e00-41ce-9727-5d906627c1c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.475431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:24.482915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e8c53-8e00-41ce-9727-5d906627c1c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:24.482915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-e03e8c53-8e00-41ce-9727-5d906627c1c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:24.482915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:24.482915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:24.482915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:24.482915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:24.482915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:24.482915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:24.482915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:24.482915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:24.482915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:24.482915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:24.482915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:24.482915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:24.487570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:24.489519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:24.489519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:24.489519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:24.489519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:24.489519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:24.489519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:24.489519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:24.489519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:24.489519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:24.489519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:24.489519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:24.489519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:24.489519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:24.489519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:24.489519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 114/227] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:24.490251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e8c53-8e00-41ce-9727-5d906627c1c6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:24.503914 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:24.599914 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e8c53-8e00-41ce-9727-5d906627c1c6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:24.636549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3836d0-9757-4a0a-90c7-11a8eb8fc6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:24.637484 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3836d0-9757-4a0a-90c7-11a8eb8fc6e0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.637982 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3836d0-9757-4a0a-90c7-11a8eb8fc6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.638446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3836d0-9757-4a0a-90c7-11a8eb8fc6e0 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.639900 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3836d0-9757-4a0a-90c7-11a8eb8fc6e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:24.640414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3836d0-9757-4a0a-90c7-11a8eb8fc6e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:24.642006 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3836d0-9757-4a0a-90c7-11a8eb8fc6e0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:24.651144 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 115/228] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:24.657102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3836d0-9757-4a0a-90c7-11a8eb8fc6e0 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:24.668869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54a8b8-2ac7-4d2a-b27c-df1abdc5957f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:24.737127 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb7714-14a1-4cfa-9518-660c9cce7301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:24.738083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb7714-14a1-4cfa-9518-660c9cce7301 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.738561 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb7714-14a1-4cfa-9518-660c9cce7301 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.739331 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb7714-14a1-4cfa-9518-660c9cce7301 None admin] PATH_INFO: `/v3/users/32db92767a194b7196757e0f4d802bd6` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.740675 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb7714-14a1-4cfa-9518-660c9cce7301 None admin] RBAC: Authorizing `identity:get_user(user_id=32db92767a194b7196757e0f4d802bd6)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:24.742378 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb7714-14a1-4cfa-9518-660c9cce7301 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:24.744214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 114/229] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:24 2022] GET /identity/v3/users/32db92767a194b7196757e0f4d802bd6 => generated 283 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:24.749824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb7714-14a1-4cfa-9518-660c9cce7301 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:24.795244 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:24.866824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb7714-14a1-4cfa-9518-660c9cce7301 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:24.893827 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:24.897059 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3836d0-9757-4a0a-90c7-11a8eb8fc6e0 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:24.907568 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3226a-02dd-48f4-88e0-96f4e4f52451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:24.908535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3226a-02dd-48f4-88e0-96f4e4f52451 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.909117 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3226a-02dd-48f4-88e0-96f4e4f52451 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.909583 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3226a-02dd-48f4-88e0-96f4e4f52451 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.911044 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3226a-02dd-48f4-88e0-96f4e4f52451 None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:24.912704 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3226a-02dd-48f4-88e0-96f4e4f52451 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:24.915730 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 115/230] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:24 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:24.920851 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3226a-02dd-48f4-88e0-96f4e4f52451 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:24.952959 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de0a3a-1e9b-404c-9390-2f419faa0f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:24.954010 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de0a3a-1e9b-404c-9390-2f419faa0f3f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.954763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de0a3a-1e9b-404c-9390-2f419faa0f3f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.955308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de0a3a-1e9b-404c-9390-2f419faa0f3f None admin] PATH_INFO: `/v3/users/32db92767a194b7196757e0f4d802bd6` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.956804 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de0a3a-1e9b-404c-9390-2f419faa0f3f None admin] RBAC: Authorizing `identity:get_user(user_id=32db92767a194b7196757e0f4d802bd6)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:24.958504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de0a3a-1e9b-404c-9390-2f419faa0f3f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:24.961410 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 116/231] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:24 2022] GET /identity/v3/users/32db92767a194b7196757e0f4d802bd6 => generated 283 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:24.969442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de0a3a-1e9b-404c-9390-2f419faa0f3f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:25.090237 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3226a-02dd-48f4-88e0-96f4e4f52451 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:25.134753 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de0a3a-1e9b-404c-9390-2f419faa0f3f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:25.157032 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ce6d4-e3e8-43e5-82eb-61a438068cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:25.157801 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ce6d4-e3e8-43e5-82eb-61a438068cad None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.157966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ce6d4-e3e8-43e5-82eb-61a438068cad None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.158244 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ce6d4-e3e8-43e5-82eb-61a438068cad None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0/users/32db92767a194b7196757e0f4d802bd6/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.164314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ce6d4-e3e8-43e5-82eb-61a438068cad None admin] RBAC: Authorizing `identity:create_grant(project_id=8c60d15f044842a0bc451a2e001075f0, user_id=32db92767a194b7196757e0f4d802bd6, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:25.165823 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ce6d4-e3e8-43e5-82eb-61a438068cad None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:25.173910 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18816721-ce72-4376-84b9-0973abc56331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:25.174802 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18816721-ce72-4376-84b9-0973abc56331 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.175349 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18816721-ce72-4376-84b9-0973abc56331 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.175835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18816721-ce72-4376-84b9-0973abc56331 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.177171 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18816721-ce72-4376-84b9-0973abc56331 None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:25.179093 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18816721-ce72-4376-84b9-0973abc56331 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:25.181240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 117/232] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:24 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:25.188017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18816721-ce72-4376-84b9-0973abc56331 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:25.191812 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:25.249597 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 116/233] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:24 2022] PUT /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0/users/32db92767a194b7196757e0f4d802bd6/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:25.309873 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18816721-ce72-4376-84b9-0973abc56331 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:25.349535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34291b94-636e-4118-a9d8-dac763c0dfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:25.350421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34291b94-636e-4118-a9d8-dac763c0dfa1 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.350838 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34291b94-636e-4118-a9d8-dac763c0dfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.351294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34291b94-636e-4118-a9d8-dac763c0dfa1 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0/users/32db92767a194b7196757e0f4d802bd6/roles/1a62bc555eb345fc8679cfa677069fd7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.353781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34291b94-636e-4118-a9d8-dac763c0dfa1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c60d15f044842a0bc451a2e001075f0, user_id=32db92767a194b7196757e0f4d802bd6, role_id=1a62bc555eb345fc8679cfa677069fd7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:25.355492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34291b94-636e-4118-a9d8-dac763c0dfa1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:25.489972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 118/234] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:25 2022] PUT /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0/users/32db92767a194b7196757e0f4d802bd6/roles/1a62bc555eb345fc8679cfa677069fd7 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:25.607432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58941a42-2cfb-4cfa-9ba3-f7d04442f492 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.608057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58941a42-2cfb-4cfa-9ba3-f7d04442f492 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.608506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58941a42-2cfb-4cfa-9ba3-f7d04442f492 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.610316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 117/235] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:25.616500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0c0d0-0207-4a5a-9d42-230e2f69a0f1 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.616784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0c0d0-0207-4a5a-9d42-230e2f69a0f1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.617062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0c0d0-0207-4a5a-9d42-230e2f69a0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.638767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-aee0c0d0-0207-4a5a-9d42-230e2f69a0f1 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:25.640649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-aee0c0d0-0207-4a5a-9d42-230e2f69a0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:25.713254 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:25.723182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 119/236] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:25.876149 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1913a3c-1a44-46fd-a781-5d88c6c123e4 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.876487 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1913a3c-1a44-46fd-a781-5d88c6c123e4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.876677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1913a3c-1a44-46fd-a781-5d88c6c123e4 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.877954 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 118/237] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:25.883851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fd0f4-d06f-4715-ac1b-48f456a4e13e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.884418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fd0f4-d06f-4715-ac1b-48f456a4e13e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.884879 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fd0f4-d06f-4715-ac1b-48f456a4e13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.911913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-339fd0f4-d06f-4715-ac1b-48f456a4e13e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:25.913486 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-339fd0f4-d06f-4715-ac1b-48f456a4e13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:26.001273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 120/238] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:26.025485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1913a3c-1a44-46fd-a781-5d88c6c123e4 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:26.104966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:26.166981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1913a3c-1a44-46fd-a781-5d88c6c123e4 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:26.216088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a3295-0c43-4f4f-9374-c761cc347ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:26.216722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a3295-0c43-4f4f-9374-c761cc347ef1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.217009 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a3295-0c43-4f4f-9374-c761cc347ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.217291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a3295-0c43-4f4f-9374-c761cc347ef1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.231998 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1a3295-0c43-4f4f-9374-c761cc347ef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:26.237850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-fa1a3295-0c43-4f4f-9374-c761cc347ef1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:26.237850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:26.237850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:26.237850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:26.237850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:26.237850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:26.237850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:26.237850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:26.237850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:26.237850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:26.237850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:26.237850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:26.237850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:26.237850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:26.237850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:26.237850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:26.238720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:26.238720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:26.238720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:26.238720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:26.238720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:26.238720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:26.238720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:26.238720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:26.238720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:26.238720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:26.238720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:26.238720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:26.238720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:26.238720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:26.238720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:26.238720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:26.239524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:26.239524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:26.239524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:26.239524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:26.239524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:26.239524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:26.239524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:26.239524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:26.239524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:26.239524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:26.239524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:26.239524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:26.240103 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 119/239] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:26.245002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fd0f4-d06f-4715-ac1b-48f456a4e13e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:26.283650 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93d1e8-2e13-43e8-bd08-05c02bd419ce None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.283987 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93d1e8-2e13-43e8-bd08-05c02bd419ce None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.284278 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93d1e8-2e13-43e8-bd08-05c02bd419ce None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.285839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 120/240] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:26.290740 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0af24d-c293-4773-a3b6-ea1308833da1 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.291048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0af24d-c293-4773-a3b6-ea1308833da1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.291325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0af24d-c293-4773-a3b6-ea1308833da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.318373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-df0af24d-c293-4773-a3b6-ea1308833da1 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:26.320341 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-df0af24d-c293-4773-a3b6-ea1308833da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:26.371414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fd0f4-d06f-4715-ac1b-48f456a4e13e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:26.400601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:26.418333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8caeb-d343-413c-aba2-36da160b6ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:26.418934 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8caeb-d343-413c-aba2-36da160b6ffc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.419161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8caeb-d343-413c-aba2-36da160b6ffc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.419430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8caeb-d343-413c-aba2-36da160b6ffc None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.420086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8caeb-d343-413c-aba2-36da160b6ffc None admin] RBAC: Adding query filter params (name=reader) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:26.421028 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8caeb-d343-413c-aba2-36da160b6ffc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:26.422283 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8caeb-d343-413c-aba2-36da160b6ffc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:26.424861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 121/241] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:26.426120 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0af24d-c293-4773-a3b6-ea1308833da1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:26.430718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 121/242] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:26.437107 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8caeb-d343-413c-aba2-36da160b6ffc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:26.485110 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:26.565177 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8caeb-d343-413c-aba2-36da160b6ffc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:26.572913 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0af24d-c293-4773-a3b6-ea1308833da1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:26.614572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289c50d-e429-401b-8111-053c90c12178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:26.615633 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289c50d-e429-401b-8111-053c90c12178 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.616214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289c50d-e429-401b-8111-053c90c12178 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.616652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289c50d-e429-401b-8111-053c90c12178 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.618191 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289c50d-e429-401b-8111-053c90c12178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:26.623986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289c50d-e429-401b-8111-053c90c12178 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:26.625795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 122/243] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:26.632845 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289c50d-e429-401b-8111-053c90c12178 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:26.646155 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406617c7-b89b-4738-8de4-c7387c99e9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:26.647093 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406617c7-b89b-4738-8de4-c7387c99e9f6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.647510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406617c7-b89b-4738-8de4-c7387c99e9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.647920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406617c7-b89b-4738-8de4-c7387c99e9f6 None admin] PATH_INFO: `/v3/users/2f0bf6dbb8ad4c9db9d4e5c64d63d834` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.673043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406617c7-b89b-4738-8de4-c7387c99e9f6 None admin] RBAC: Authorizing `identity:get_user(user_id=2f0bf6dbb8ad4c9db9d4e5c64d63d834)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:26.675026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406617c7-b89b-4738-8de4-c7387c99e9f6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:26.678032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 122/244] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:26 2022] GET /identity/v3/users/2f0bf6dbb8ad4c9db9d4e5c64d63d834 => generated 297 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:26.686451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406617c7-b89b-4738-8de4-c7387c99e9f6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:26.774729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:26.818580 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289c50d-e429-401b-8111-053c90c12178 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:26.837693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:26.868569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406617c7-b89b-4738-8de4-c7387c99e9f6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:26.869082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed35cbd-3e83-4611-af77-a0253c56584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:26.869807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed35cbd-3e83-4611-af77-a0253c56584d None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.870179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed35cbd-3e83-4611-af77-a0253c56584d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.870539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed35cbd-3e83-4611-af77-a0253c56584d None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.871564 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed35cbd-3e83-4611-af77-a0253c56584d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:26.874063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed35cbd-3e83-4611-af77-a0253c56584d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:26.904271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0d5c2-dbb5-4642-ad54-70ae0aa38e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:26.905402 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0d5c2-dbb5-4642-ad54-70ae0aa38e93 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.905678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0d5c2-dbb5-4642-ad54-70ae0aa38e93 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.905943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0d5c2-dbb5-4642-ad54-70ae0aa38e93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.977183 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0d5c2-dbb5-4642-ad54-70ae0aa38e93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:26.978546 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0d5c2-dbb5-4642-ad54-70ae0aa38e93 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:26.990976 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 123/245] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:37:26 2022] POST /identity/v3/users => generated 293 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:37:26.990976 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed35cbd-3e83-4611-af77-a0253c56584d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:27.066765 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:27.176191 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 123/246] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:27.182481 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0d5c2-dbb5-4642-ad54-70ae0aa38e93 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:27.218575 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed35cbd-3e83-4611-af77-a0253c56584d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:27.271194 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796aef08-fc8f-4c8e-9250-6562415cc74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:27.272933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796aef08-fc8f-4c8e-9250-6562415cc74a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:27.273534 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796aef08-fc8f-4c8e-9250-6562415cc74a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:27.274069 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796aef08-fc8f-4c8e-9250-6562415cc74a None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:27.290870 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796aef08-fc8f-4c8e-9250-6562415cc74a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:27.298755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-796aef08-fc8f-4c8e-9250-6562415cc74a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:27.298755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:27.298755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:27.298755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:27.298755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:27.298755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:27.298755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:27.298755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:27.298755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:27.298755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:27.298755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:27.298755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:27.298755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:27.298755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:27.298755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:27.298755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:27.299629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:27.299629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:27.299629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:27.299629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:27.299629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:27.299629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:27.299629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:27.299629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:27.299629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:27.299629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:27.299629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:27.299629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:27.299629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:27.299629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:27.299629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:27.299629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:27.300403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:27.300403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:27.300403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:27.300403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:27.300403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:27.300403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:27.300403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:27.300403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:27.300403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:27.300403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:27.300403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:27.300403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:27.303614 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 124/247] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:27.314873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796aef08-fc8f-4c8e-9250-6562415cc74a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:27.337955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0d5c2-dbb5-4642-ad54-70ae0aa38e93 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:27.346136 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:27.381031 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e138dd-0ee1-4ba1-bebf-665292abfcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:27.381721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e138dd-0ee1-4ba1-bebf-665292abfcfc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:27.382005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e138dd-0ee1-4ba1-bebf-665292abfcfc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:27.382343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e138dd-0ee1-4ba1-bebf-665292abfcfc None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:27.383546 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e138dd-0ee1-4ba1-bebf-665292abfcfc None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:27.385134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e138dd-0ee1-4ba1-bebf-665292abfcfc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:27.387030 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 124/248] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:27 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:27.392900 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e138dd-0ee1-4ba1-bebf-665292abfcfc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:27.411564 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:27.433179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796aef08-fc8f-4c8e-9250-6562415cc74a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:27.472312 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bda9c3-cdfd-4f04-a050-d6fac9786250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:27.473264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bda9c3-cdfd-4f04-a050-d6fac9786250 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:27.473770 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bda9c3-cdfd-4f04-a050-d6fac9786250 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:27.474243 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bda9c3-cdfd-4f04-a050-d6fac9786250 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:27.475182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bda9c3-cdfd-4f04-a050-d6fac9786250 None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:27.475698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bda9c3-cdfd-4f04-a050-d6fac9786250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:27.477411 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bda9c3-cdfd-4f04-a050-d6fac9786250 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:27.488000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 125/249] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:27.498606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bda9c3-cdfd-4f04-a050-d6fac9786250 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:27.509926 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e138dd-0ee1-4ba1-bebf-665292abfcfc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:27.547872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93818e9-11e1-4143-91b9-10439534873a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:27.548396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93818e9-11e1-4143-91b9-10439534873a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:27.548881 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93818e9-11e1-4143-91b9-10439534873a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:27.549167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93818e9-11e1-4143-91b9-10439534873a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:27.549993 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93818e9-11e1-4143-91b9-10439534873a None admin] RBAC: Adding query filter params (role.id=d3fb1e3ec057480992710ad61b6cd1ed, scope.project.id=8c60d15f044842a0bc451a2e001075f0, user.id=2f0bf6dbb8ad4c9db9d4e5c64d63d834) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:27.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93818e9-11e1-4143-91b9-10439534873a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:27.552918 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93818e9-11e1-4143-91b9-10439534873a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:27.560411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 125/250] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:27 2022] GET /identity/v3/role_assignments?user.id=2f0bf6dbb8ad4c9db9d4e5c64d63d834&scope.project.id=8c60d15f044842a0bc451a2e001075f0&role.id=d3fb1e3ec057480992710ad61b6cd1ed => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:27.600781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:27.615269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bda9c3-cdfd-4f04-a050-d6fac9786250 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:27.656409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f0b9c-a460-4257-95b2-5fa2a932ba51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:27.657064 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f0b9c-a460-4257-95b2-5fa2a932ba51 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:27.657347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f0b9c-a460-4257-95b2-5fa2a932ba51 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:27.657622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f0b9c-a460-4257-95b2-5fa2a932ba51 None admin] PATH_INFO: `/v3/users/fb74804ffc39429e9a10ef19845a532d` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:27.658829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f0b9c-a460-4257-95b2-5fa2a932ba51 None admin] RBAC: Authorizing `identity:get_user(user_id=fb74804ffc39429e9a10ef19845a532d)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:27.660421 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f0b9c-a460-4257-95b2-5fa2a932ba51 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:27.662414 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 126/251] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:27 2022] GET /identity/v3/users/fb74804ffc39429e9a10ef19845a532d => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:27.668367 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93818e9-11e1-4143-91b9-10439534873a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:27.768160 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:27.796340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93818e9-11e1-4143-91b9-10439534873a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:27.836706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c931edd-00c5-4bc9-8312-32acca645150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:27.837259 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c931edd-00c5-4bc9-8312-32acca645150 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:27.837539 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c931edd-00c5-4bc9-8312-32acca645150 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:27.837810 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c931edd-00c5-4bc9-8312-32acca645150 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:27.838996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c931edd-00c5-4bc9-8312-32acca645150 None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:27.840455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c931edd-00c5-4bc9-8312-32acca645150 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:27.843020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 126/252] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:27 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:27.848831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f0b9c-a460-4257-95b2-5fa2a932ba51 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:27.955662 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f0b9c-a460-4257-95b2-5fa2a932ba51 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:27.994752 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485c152-9563-4edf-abe2-d5706dc1e551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:27.995481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485c152-9563-4edf-abe2-d5706dc1e551 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:27.995760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485c152-9563-4edf-abe2-d5706dc1e551 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:27.996034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485c152-9563-4edf-abe2-d5706dc1e551 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0/users/fb74804ffc39429e9a10ef19845a532d/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:27.998186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485c152-9563-4edf-abe2-d5706dc1e551 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c60d15f044842a0bc451a2e001075f0, user_id=fb74804ffc39429e9a10ef19845a532d, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:27.999792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485c152-9563-4edf-abe2-d5706dc1e551 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:28.074484 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 127/253] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:27 2022] PUT /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0/users/fb74804ffc39429e9a10ef19845a532d/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:28.132847 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1d39c-c0b9-4d8d-a013-e87ec557c53c None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:28.133494 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1d39c-c0b9-4d8d-a013-e87ec557c53c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:28.133989 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1d39c-c0b9-4d8d-a013-e87ec557c53c None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:28.135863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 127/254] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:28.142430 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7d3ed-b17b-4896-a8b3-ca7be5f12fef None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:28.142735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7d3ed-b17b-4896-a8b3-ca7be5f12fef None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:28.143002 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7d3ed-b17b-4896-a8b3-ca7be5f12fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:28.165426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-5ea7d3ed-b17b-4896-a8b3-ca7be5f12fef None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:28.166653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5ea7d3ed-b17b-4896-a8b3-ca7be5f12fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:28.182673 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:28.236866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cafb4e-1460-4ef7-8c37-6102334e8725 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:28.237217 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cafb4e-1460-4ef7-8c37-6102334e8725 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:28.237405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cafb4e-1460-4ef7-8c37-6102334e8725 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:28.238994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 128/255] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:28.252764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1790e-2d0f-48a1-b2dc-280faeca2baa None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:28.253060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1790e-2d0f-48a1-b2dc-280faeca2baa None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:28.253300 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1790e-2d0f-48a1-b2dc-280faeca2baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:28.279724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 128/256] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:28.286714 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8da1790e-2d0f-48a1-b2dc-280faeca2baa None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:28.288254 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8da1790e-2d0f-48a1-b2dc-280faeca2baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:28.370608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 129/257] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:28.640276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7d3ed-b17b-4896-a8b3-ca7be5f12fef None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:28.760951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7d3ed-b17b-4896-a8b3-ca7be5f12fef None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:28.806413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8fde3-6d78-4e2f-8d3e-7b55e05cab75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:28.807083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8fde3-6d78-4e2f-8d3e-7b55e05cab75 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:28.807324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1790e-2d0f-48a1-b2dc-280faeca2baa None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:28.807473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8fde3-6d78-4e2f-8d3e-7b55e05cab75 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:28.807660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8fde3-6d78-4e2f-8d3e-7b55e05cab75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:28.834065 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8fde3-6d78-4e2f-8d3e-7b55e05cab75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:28.844197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-dcc8fde3-6d78-4e2f-8d3e-7b55e05cab75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:28.844197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:28.844197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:28.844197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:28.844197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:28.844197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:28.844197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:28.844197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:28.844197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:28.844197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:28.844197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:28.844197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:28.844197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:28.844197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:28.844197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:28.844197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:28.845087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:28.845087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:28.845087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:28.845087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:28.845087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:28.845087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:28.845087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:28.845087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:28.845087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:28.845087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:28.845087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:28.845087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:28.845087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:28.845087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:28.845087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:28.845087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:28.845856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:28.845856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:28.845856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:28.845856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:28.845856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:28.845856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:28.845856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:28.845856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:28.845856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:28.845856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:28.845856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:28.845856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:28.851971 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 129/258] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:28.855587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8fde3-6d78-4e2f-8d3e-7b55e05cab75 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:28.875393 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:28.978755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:29.028399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1790e-2d0f-48a1-b2dc-280faeca2baa None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.083293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7081f0-5568-4439-903e-b17426370b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.084404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7081f0-5568-4439-903e-b17426370b02 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.084693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7081f0-5568-4439-903e-b17426370b02 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.084964 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7081f0-5568-4439-903e-b17426370b02 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.105188 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7081f0-5568-4439-903e-b17426370b02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:29.110181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-cf7081f0-5568-4439-903e-b17426370b02 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:29.110181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:29.110181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:29.110181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:29.110181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:29.110181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:29.110181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:29.110181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:29.110181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:29.110181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:29.110181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:29.110181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:29.110181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:29.110181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:29.110181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:29.110181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:29.112245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:29.112245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:29.112245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:29.112245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:29.112245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:29.112245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:29.112245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:29.112245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:29.112245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:29.112245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:29.112245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:29.112245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:29.112245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:29.112245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:29.112245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:29.112245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:29.114138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:29.114138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:29.114138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:29.114138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:29.114138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:29.114138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:29.114138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:29.114138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:29.114138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:29.114138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:29.114138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:29.114138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:29.114138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 130/259] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:29.115827 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8fde3-6d78-4e2f-8d3e-7b55e05cab75 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.132627 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7081f0-5568-4439-903e-b17426370b02 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.203690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3179c9-8dcb-48d2-88cb-8c09efb05365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.204320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3179c9-8dcb-48d2-88cb-8c09efb05365 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.204598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3179c9-8dcb-48d2-88cb-8c09efb05365 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.204868 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3179c9-8dcb-48d2-88cb-8c09efb05365 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.205551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3179c9-8dcb-48d2-88cb-8c09efb05365 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:29.205851 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3179c9-8dcb-48d2-88cb-8c09efb05365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:29.207319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3179c9-8dcb-48d2-88cb-8c09efb05365 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:29.226397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 130/260] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:29.231891 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3179c9-8dcb-48d2-88cb-8c09efb05365 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.321829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7081f0-5568-4439-903e-b17426370b02 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.385776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237afac-6cc2-4f52-90b3-5dc2e83dc3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.386526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237afac-6cc2-4f52-90b3-5dc2e83dc3da None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.386683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237afac-6cc2-4f52-90b3-5dc2e83dc3da None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.386934 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237afac-6cc2-4f52-90b3-5dc2e83dc3da None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.387613 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237afac-6cc2-4f52-90b3-5dc2e83dc3da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:29.388580 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237afac-6cc2-4f52-90b3-5dc2e83dc3da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:29.390588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237afac-6cc2-4f52-90b3-5dc2e83dc3da None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:29.392799 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:29.401916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 131/261] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:29.403280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:29.409663 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237afac-6cc2-4f52-90b3-5dc2e83dc3da None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.496893 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3179c9-8dcb-48d2-88cb-8c09efb05365 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.593130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e3f8b-d89d-409c-a2e3-20530999f657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.593694 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e3f8b-d89d-409c-a2e3-20530999f657 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.593981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e3f8b-d89d-409c-a2e3-20530999f657 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.594258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e3f8b-d89d-409c-a2e3-20530999f657 None admin] PATH_INFO: `/v3/users/32db92767a194b7196757e0f4d802bd6` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.599579 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e3f8b-d89d-409c-a2e3-20530999f657 None admin] RBAC: Authorizing `identity:get_user(user_id=32db92767a194b7196757e0f4d802bd6)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:29.601358 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e3f8b-d89d-409c-a2e3-20530999f657 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:29.603052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 131/262] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:29 2022] GET /identity/v3/users/32db92767a194b7196757e0f4d802bd6 => generated 283 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:29.606656 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237afac-6cc2-4f52-90b3-5dc2e83dc3da None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.609313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e3f8b-d89d-409c-a2e3-20530999f657 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.674790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f765b1-228d-4c6e-b8a1-3ababe179c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.675462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f765b1-228d-4c6e-b8a1-3ababe179c9f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.675745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f765b1-228d-4c6e-b8a1-3ababe179c9f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.676022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f765b1-228d-4c6e-b8a1-3ababe179c9f None admin] PATH_INFO: `/v3/users/32db92767a194b7196757e0f4d802bd6` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.680475 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f765b1-228d-4c6e-b8a1-3ababe179c9f None admin] RBAC: Authorizing `identity:get_user(user_id=32db92767a194b7196757e0f4d802bd6)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:29.682021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f765b1-228d-4c6e-b8a1-3ababe179c9f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:29.684571 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 132/263] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:29 2022] GET /identity/v3/users/32db92767a194b7196757e0f4d802bd6 => generated 283 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:29.691444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f765b1-228d-4c6e-b8a1-3ababe179c9f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.841255 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e3f8b-d89d-409c-a2e3-20530999f657 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.845095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:29.898539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455038d3-1889-42e4-a84a-a5916a6e93f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.899274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455038d3-1889-42e4-a84a-a5916a6e93f4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.899558 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455038d3-1889-42e4-a84a-a5916a6e93f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.899831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455038d3-1889-42e4-a84a-a5916a6e93f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.906682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:29.949047 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f765b1-228d-4c6e-b8a1-3ababe179c9f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:30.005577 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455038d3-1889-42e4-a84a-a5916a6e93f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.007403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455038d3-1889-42e4-a84a-a5916a6e93f4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.047605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7836d2-8ac9-4b45-98ef-c4418565edc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.048367 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7836d2-8ac9-4b45-98ef-c4418565edc7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.048774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7836d2-8ac9-4b45-98ef-c4418565edc7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.049136 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7836d2-8ac9-4b45-98ef-c4418565edc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.152141 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7836d2-8ac9-4b45-98ef-c4418565edc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.153980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7836d2-8ac9-4b45-98ef-c4418565edc7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.167497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 132/264] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:30.173880 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455038d3-1889-42e4-a84a-a5916a6e93f4 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:30.226336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:30.254927 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:30.285627 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 133/265] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:30.291244 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7836d2-8ac9-4b45-98ef-c4418565edc7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:30.329630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455038d3-1889-42e4-a84a-a5916a6e93f4 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:30.400531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38ac65-e39f-4dd0-9163-40ff4869f928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.401557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ac65-e39f-4dd0-9163-40ff4869f928 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.402042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ac65-e39f-4dd0-9163-40ff4869f928 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.402571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38ac65-e39f-4dd0-9163-40ff4869f928 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.407583 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38ac65-e39f-4dd0-9163-40ff4869f928 None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.409749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38ac65-e39f-4dd0-9163-40ff4869f928 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.411987 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 133/266] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:30 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:30.417584 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38ac65-e39f-4dd0-9163-40ff4869f928 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:30.504961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7836d2-8ac9-4b45-98ef-c4418565edc7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:30.591803 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267566e-1d34-4a98-8319-1375cba0dd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.592436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267566e-1d34-4a98-8319-1375cba0dd98 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.593344 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267566e-1d34-4a98-8319-1375cba0dd98 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.593620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267566e-1d34-4a98-8319-1375cba0dd98 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.595304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267566e-1d34-4a98-8319-1375cba0dd98 None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.596771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267566e-1d34-4a98-8319-1375cba0dd98 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.598406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 134/267] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:30 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:30.605235 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267566e-1d34-4a98-8319-1375cba0dd98 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:30.649212 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38ac65-e39f-4dd0-9163-40ff4869f928 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:30.656527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:30.663232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:30.733688 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267566e-1d34-4a98-8319-1375cba0dd98 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:30.735658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fde434-cc33-4278-a289-d4ae799d473b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.736253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fde434-cc33-4278-a289-d4ae799d473b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.736534 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fde434-cc33-4278-a289-d4ae799d473b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.736805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fde434-cc33-4278-a289-d4ae799d473b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.740936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fde434-cc33-4278-a289-d4ae799d473b None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=8c60d15f044842a0bc451a2e001075f0, user.id=32db92767a194b7196757e0f4d802bd6) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:30.740936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fde434-cc33-4278-a289-d4ae799d473b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.740936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fde434-cc33-4278-a289-d4ae799d473b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.745814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 134/268] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:30 2022] GET /identity/v3/role_assignments?user.id=32db92767a194b7196757e0f4d802bd6&scope.project.id=8c60d15f044842a0bc451a2e001075f0&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 608 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:30.747344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797e767-e744-4f66-95b8-7f1a7231992c None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.747820 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797e767-e744-4f66-95b8-7f1a7231992c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.748275 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797e767-e744-4f66-95b8-7f1a7231992c None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.750231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 135/269] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:30.751599 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de514863-5c56-400b-bc10-5316a92d905c None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.752122 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de514863-5c56-400b-bc10-5316a92d905c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.752599 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de514863-5c56-400b-bc10-5316a92d905c None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.754157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 136/270] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:30.755904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376eaa2b-05be-4513-a74b-973feb5e53e3 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.756419 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376eaa2b-05be-4513-a74b-973feb5e53e3 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.757074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376eaa2b-05be-4513-a74b-973feb5e53e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.772364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211fc4b-a1a1-47da-8065-f5cfc8bbaf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.772972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211fc4b-a1a1-47da-8065-f5cfc8bbaf40 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.773244 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211fc4b-a1a1-47da-8065-f5cfc8bbaf40 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.773514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211fc4b-a1a1-47da-8065-f5cfc8bbaf40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.774240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211fc4b-a1a1-47da-8065-f5cfc8bbaf40 None admin] RBAC: Adding query filter params (role.id=1a62bc555eb345fc8679cfa677069fd7, scope.project.id=8c60d15f044842a0bc451a2e001075f0, user.id=32db92767a194b7196757e0f4d802bd6) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:30.774544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211fc4b-a1a1-47da-8065-f5cfc8bbaf40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.776051 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211fc4b-a1a1-47da-8065-f5cfc8bbaf40 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.778076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-376eaa2b-05be-4513-a74b-973feb5e53e3 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:30.779686 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-376eaa2b-05be-4513-a74b-973feb5e53e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:30.782848 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 135/271] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:30 2022] GET /identity/v3/role_assignments?user.id=32db92767a194b7196757e0f4d802bd6&scope.project.id=8c60d15f044842a0bc451a2e001075f0&role.id=1a62bc555eb345fc8679cfa677069fd7 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:30.784170 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10bfe5-284d-437b-abd9-41f67271519c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.784456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10bfe5-284d-437b-abd9-41f67271519c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.784720 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10bfe5-284d-437b-abd9-41f67271519c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.804857 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-6a10bfe5-284d-437b-abd9-41f67271519c None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:30.806494 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6a10bfe5-284d-437b-abd9-41f67271519c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:30.863194 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 137/272] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:30.893864 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 136/273] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:31.191808 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376eaa2b-05be-4513-a74b-973feb5e53e3 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:31.272126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10bfe5-284d-437b-abd9-41f67271519c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:31.314927 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:31.316367 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:31.320394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376eaa2b-05be-4513-a74b-973feb5e53e3 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:31.358336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb8fd1-f61a-4541-b9f2-87c34b621a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:31.359708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb8fd1-f61a-4541-b9f2-87c34b621a5e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:31.359996 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb8fd1-f61a-4541-b9f2-87c34b621a5e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:31.360270 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb8fd1-f61a-4541-b9f2-87c34b621a5e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:31.375855 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb8fd1-f61a-4541-b9f2-87c34b621a5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:31.381526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-c4fb8fd1-f61a-4541-b9f2-87c34b621a5e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:31.381526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:31.381526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:31.381526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:31.381526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:31.381526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:31.381526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:31.381526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:31.381526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:31.381526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:31.381526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:31.381526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:31.381526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:31.381526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:31.381526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:31.381526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:31.401825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:31.401825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:31.401825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:31.401825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:31.401825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:31.401825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:31.401825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:31.401825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:31.401825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:31.401825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:31.401825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:31.401825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:31.401825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:31.401825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:31.401825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:31.401825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:31.402715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:31.402715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:31.402715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:31.402715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:31.402715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:31.402715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:31.402715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:31.402715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:31.402715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:31.402715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:31.402715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:31.402715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:31.402715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 138/274] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:31.402715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121b290-1b39-471b-9627-8b1a8d4b1114 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:31.403527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121b290-1b39-471b-9627-8b1a8d4b1114 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:31.403527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121b290-1b39-471b-9627-8b1a8d4b1114 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:31.403527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 139/275] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:31.403527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121b290-1b39-471b-9627-8b1a8d4b1114 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:31.403741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10bfe5-284d-437b-abd9-41f67271519c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:31.441607 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662b7e6-52de-4359-ba48-71c4c3c211b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:31.442286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662b7e6-52de-4359-ba48-71c4c3c211b4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:31.442516 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662b7e6-52de-4359-ba48-71c4c3c211b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:31.442787 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662b7e6-52de-4359-ba48-71c4c3c211b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:31.453961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662b7e6-52de-4359-ba48-71c4c3c211b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:31.458684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-c662b7e6-52de-4359-ba48-71c4c3c211b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:31.458684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:31.458684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:31.458684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:31.458684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:31.458684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:31.458684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:31.458684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:31.458684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:31.458684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:31.458684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:31.458684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:31.458684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:31.458684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:31.458684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:31.458684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:31.459591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:31.459591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:31.459591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:31.459591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:31.459591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:31.459591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:31.459591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:31.459591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:31.459591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:31.459591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:31.459591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:31.459591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:31.459591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:31.459591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:31.459591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:31.459591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:31.460442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:31.460442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:31.460442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:31.460442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:31.460442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:31.460442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:31.460442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:31.460442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:31.460442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:31.460442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:31.460442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:31.460442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:31.460442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 137/276] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:37:31.463437 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2137289-eb97-472e-86e1-87881040a977 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:31.463738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2137289-eb97-472e-86e1-87881040a977 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:31.463995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2137289-eb97-472e-86e1-87881040a977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:31.484069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b2137289-eb97-472e-86e1-87881040a977 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:31.485264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b2137289-eb97-472e-86e1-87881040a977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:31.505528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121b290-1b39-471b-9627-8b1a8d4b1114 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:31.539153 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b1756-b69a-4c3c-9cf8-538fbbf47953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:31.539753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b1756-b69a-4c3c-9cf8-538fbbf47953 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:31.540027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b1756-b69a-4c3c-9cf8-538fbbf47953 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:31.540298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b1756-b69a-4c3c-9cf8-538fbbf47953 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:31.540946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b1756-b69a-4c3c-9cf8-538fbbf47953 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:31.541243 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b1756-b69a-4c3c-9cf8-538fbbf47953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:31.542627 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b1756-b69a-4c3c-9cf8-538fbbf47953 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:31.551404 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 140/277] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:31.552536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f111-7ddb-468d-99b4-fed2d2262092 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:31.552814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f111-7ddb-468d-99b4-fed2d2262092 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:31.553080 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f111-7ddb-468d-99b4-fed2d2262092 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:31.554276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 141/278] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:31.555750 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3f111-7ddb-468d-99b4-fed2d2262092 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:31.557890 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 138/279] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:31.559830 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2137289-eb97-472e-86e1-87881040a977 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:31.579042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:31.585842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:31.666344 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2137289-eb97-472e-86e1-87881040a977 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:31.674626 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3f111-7ddb-468d-99b4-fed2d2262092 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:31.703832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fc6ca-0317-4da2-89cc-91966e7cf106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:31.704494 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fc6ca-0317-4da2-89cc-91966e7cf106 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:31.704771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fc6ca-0317-4da2-89cc-91966e7cf106 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:31.705043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fc6ca-0317-4da2-89cc-91966e7cf106 None admin] PATH_INFO: `/v3/users/2f0bf6dbb8ad4c9db9d4e5c64d63d834` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:31.706255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50fc6ca-0317-4da2-89cc-91966e7cf106 None admin] RBAC: Authorizing `identity:get_user(user_id=2f0bf6dbb8ad4c9db9d4e5c64d63d834)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:31.707950 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50fc6ca-0317-4da2-89cc-91966e7cf106 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:31.708894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfdc0af-55c5-4e0e-b789-8087122c30bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:31.714263 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfdc0af-55c5-4e0e-b789-8087122c30bc None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:31.714523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 139/280] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:31 2022] GET /identity/v3/users/2f0bf6dbb8ad4c9db9d4e5c64d63d834 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:31.714603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfdc0af-55c5-4e0e-b789-8087122c30bc None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:31.714603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfdc0af-55c5-4e0e-b789-8087122c30bc None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:31.714603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfdc0af-55c5-4e0e-b789-8087122c30bc None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:31.714794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f9d06-e3a4-4c4a-bae0-b5d184f55f47 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:31.714865 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfdc0af-55c5-4e0e-b789-8087122c30bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:31.714956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f9d06-e3a4-4c4a-bae0-b5d184f55f47 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:31.714956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f9d06-e3a4-4c4a-bae0-b5d184f55f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:31.715075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfdc0af-55c5-4e0e-b789-8087122c30bc None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:31.723336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 142/281] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:31.728452 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfdc0af-55c5-4e0e-b789-8087122c30bc None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:31.730769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8f1f9d06-e3a4-4c4a-bae0-b5d184f55f47 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:31.732104 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8f1f9d06-e3a4-4c4a-bae0-b5d184f55f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:31.814925 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 140/282] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:31.814925 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f9d06-e3a4-4c4a-bae0-b5d184f55f47 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:31.820661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:31.831279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:31.841337 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfdc0af-55c5-4e0e-b789-8087122c30bc None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:31.879476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0414f-5e2d-493e-8a83-957701aca378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:31.880153 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0414f-5e2d-493e-8a83-957701aca378 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:31.880377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0414f-5e2d-493e-8a83-957701aca378 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:31.880646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0414f-5e2d-493e-8a83-957701aca378 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:31.881773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0414f-5e2d-493e-8a83-957701aca378 None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:31.883251 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0414f-5e2d-493e-8a83-957701aca378 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:31.887014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 143/283] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:31 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:31.887014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0414f-5e2d-493e-8a83-957701aca378 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:31.924491 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f9d06-e3a4-4c4a-bae0-b5d184f55f47 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:31.961670 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f4a36-2d40-4743-9938-1f684faa4db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:31.962823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f4a36-2d40-4743-9938-1f684faa4db0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:31.962823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f4a36-2d40-4743-9938-1f684faa4db0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:31.962823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f4a36-2d40-4743-9938-1f684faa4db0 None admin] PATH_INFO: `/v3/users/7ced57aac59747a1b7ecdc96b1449539` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:31.976921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f4a36-2d40-4743-9938-1f684faa4db0 None admin] RBAC: Authorizing `identity:get_user(user_id=7ced57aac59747a1b7ecdc96b1449539)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:31.978680 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f4a36-2d40-4743-9938-1f684faa4db0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:31.980774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 141/284] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:31 2022] GET /identity/v3/users/7ced57aac59747a1b7ecdc96b1449539 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:31.981723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f4a36-2d40-4743-9938-1f684faa4db0 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:31.997492 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0414f-5e2d-493e-8a83-957701aca378 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:32.034856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38540f7e-242b-43b3-b817-1cb68b3c5093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:32.035506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38540f7e-242b-43b3-b817-1cb68b3c5093 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:32.035784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38540f7e-242b-43b3-b817-1cb68b3c5093 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:32.036057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38540f7e-242b-43b3-b817-1cb68b3c5093 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:32.047966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38540f7e-242b-43b3-b817-1cb68b3c5093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:32.052813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-38540f7e-242b-43b3-b817-1cb68b3c5093 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:32.052813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:32.052813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:32.052813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:32.052813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:32.052813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:32.052813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:32.052813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:32.052813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:32.052813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:32.052813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:32.052813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:32.052813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:32.052813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:32.052813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:32.052813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:32.053695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:32.053695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:32.053695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:32.053695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:32.053695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:32.053695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:32.053695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:32.053695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:32.053695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:32.053695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:32.053695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:32.053695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:32.053695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:32.053695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:32.053695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:32.053695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:32.054472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:32.054472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:32.054472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:32.054472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:32.054472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:32.054472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:32.054472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:32.054472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:32.054472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:32.054472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:32.054472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:32.054472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:32.054472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 144/285] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:32.055304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38540f7e-242b-43b3-b817-1cb68b3c5093 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:32.062999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:32.073227 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:32.096102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f4a36-2d40-4743-9938-1f684faa4db0 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:32.131681 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090d497-a125-41c9-8908-11e963f99ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:32.132340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090d497-a125-41c9-8908-11e963f99ff7 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:32.132619 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090d497-a125-41c9-8908-11e963f99ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:32.132894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090d497-a125-41c9-8908-11e963f99ff7 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0/users/2f0bf6dbb8ad4c9db9d4e5c64d63d834/roles/d3fb1e3ec057480992710ad61b6cd1ed` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:32.134818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090d497-a125-41c9-8908-11e963f99ff7 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c60d15f044842a0bc451a2e001075f0, user_id=2f0bf6dbb8ad4c9db9d4e5c64d63d834, role_id=d3fb1e3ec057480992710ad61b6cd1ed)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:32.136308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090d497-a125-41c9-8908-11e963f99ff7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:32.159893 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38540f7e-242b-43b3-b817-1cb68b3c5093 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:32.192642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16413c5c-1789-400d-85a4-df4e51effc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:32.193228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16413c5c-1789-400d-85a4-df4e51effc3d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:32.193499 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16413c5c-1789-400d-85a4-df4e51effc3d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:32.193763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16413c5c-1789-400d-85a4-df4e51effc3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:32.194470 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16413c5c-1789-400d-85a4-df4e51effc3d None admin] RBAC: Adding query filter params (role.id=cb722313950945e5b147e994ec678cd1, user.id=7ced57aac59747a1b7ecdc96b1449539, scope.system=all) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:32.194766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16413c5c-1789-400d-85a4-df4e51effc3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:32.196236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16413c5c-1789-400d-85a4-df4e51effc3d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:32.200925 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 145/286] 10.222.0.91 () {56 vars in 1308 bytes} [Thu Sep 8 13:37:32 2022] GET /identity/v3/role_assignments?user.id=7ced57aac59747a1b7ecdc96b1449539&scope.system=all&role.id=cb722313950945e5b147e994ec678cd1 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:32.201689 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16413c5c-1789-400d-85a4-df4e51effc3d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:32.222607 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 142/287] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:31 2022] PUT /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0/users/2f0bf6dbb8ad4c9db9d4e5c64d63d834/roles/d3fb1e3ec057480992710ad61b6cd1ed => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:32.223408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090d497-a125-41c9-8908-11e963f99ff7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:32.328272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:32.363049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16413c5c-1789-400d-85a4-df4e51effc3d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:32.373108 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090d497-a125-41c9-8908-11e963f99ff7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:32.411196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcf48d-4f32-4de1-a440-837dcb4a75de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:32.411844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcf48d-4f32-4de1-a440-837dcb4a75de None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:32.412119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcf48d-4f32-4de1-a440-837dcb4a75de None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:32.412390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcf48d-4f32-4de1-a440-837dcb4a75de None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:32.413209 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcf48d-4f32-4de1-a440-837dcb4a75de None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:32.413510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcf48d-4f32-4de1-a440-837dcb4a75de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:32.415024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcf48d-4f32-4de1-a440-837dcb4a75de None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:32.418515 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:32.424093 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a74b48-711c-4ee3-8097-3fb206968282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:32.425120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a74b48-711c-4ee3-8097-3fb206968282 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:32.425768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 146/288] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:32.426153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a74b48-711c-4ee3-8097-3fb206968282 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:32.426650 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a74b48-711c-4ee3-8097-3fb206968282 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:32.432542 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcf48d-4f32-4de1-a440-837dcb4a75de None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:32.433352 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a74b48-711c-4ee3-8097-3fb206968282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:32.435104 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a74b48-711c-4ee3-8097-3fb206968282 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:32.437045 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 143/289] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:32.444360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a74b48-711c-4ee3-8097-3fb206968282 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:32.557169 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcf48d-4f32-4de1-a440-837dcb4a75de None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:32.585708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a74b48-711c-4ee3-8097-3fb206968282 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:32.594966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889a9d9-85d1-4d0d-94c2-3e35e2c2b32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:32.595905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889a9d9-85d1-4d0d-94c2-3e35e2c2b32d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:32.596327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889a9d9-85d1-4d0d-94c2-3e35e2c2b32d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:32.596741 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889a9d9-85d1-4d0d-94c2-3e35e2c2b32d None admin] PATH_INFO: `/v3/users/fb74804ffc39429e9a10ef19845a532d` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:32.598037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889a9d9-85d1-4d0d-94c2-3e35e2c2b32d None admin] RBAC: Authorizing `identity:get_user(user_id=fb74804ffc39429e9a10ef19845a532d)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:32.599762 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889a9d9-85d1-4d0d-94c2-3e35e2c2b32d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:32.602042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 147/290] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:32 2022] GET /identity/v3/users/fb74804ffc39429e9a10ef19845a532d => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:32.610723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889a9d9-85d1-4d0d-94c2-3e35e2c2b32d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:32.617459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:32.628315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0de1e-6894-4158-a65b-164988578aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:32.629394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0de1e-6894-4158-a65b-164988578aeb None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:32.634612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0de1e-6894-4158-a65b-164988578aeb None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:32.634612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0de1e-6894-4158-a65b-164988578aeb None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:32.634612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0de1e-6894-4158-a65b-164988578aeb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:32.634612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0de1e-6894-4158-a65b-164988578aeb None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:32.714805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889a9d9-85d1-4d0d-94c2-3e35e2c2b32d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:32.752339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 144/291] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:37:32 2022] POST /identity/v3/users => generated 293 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:32.754932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac1f37-fa4b-41d8-b11d-43de073efb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:32.755727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac1f37-fa4b-41d8-b11d-43de073efb7b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:32.756011 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac1f37-fa4b-41d8-b11d-43de073efb7b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:32.756285 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac1f37-fa4b-41d8-b11d-43de073efb7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:32.843229 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac1f37-fa4b-41d8-b11d-43de073efb7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:32.844728 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac1f37-fa4b-41d8-b11d-43de073efb7b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:32.847874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:32.960090 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 148/292] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:32.967664 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0de1e-6894-4158-a65b-164988578aeb None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:33.036783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:33.085516 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0de1e-6894-4158-a65b-164988578aeb None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:33.123187 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285e91b-7aff-4f10-8334-c89701ddf7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:33.123831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285e91b-7aff-4f10-8334-c89701ddf7a0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:33.124152 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285e91b-7aff-4f10-8334-c89701ddf7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:33.124911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285e91b-7aff-4f10-8334-c89701ddf7a0 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:33.126115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285e91b-7aff-4f10-8334-c89701ddf7a0 None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:33.127628 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285e91b-7aff-4f10-8334-c89701ddf7a0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:33.130825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 145/293] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:32 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:33.136195 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac1f37-fa4b-41d8-b11d-43de073efb7b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:33.246581 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac1f37-fa4b-41d8-b11d-43de073efb7b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:33.249709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:33.285718 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8466a9-3355-4fbc-b7ea-fb88eb4e7c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:33.286848 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8466a9-3355-4fbc-b7ea-fb88eb4e7c54 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:33.287631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8466a9-3355-4fbc-b7ea-fb88eb4e7c54 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:33.287903 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8466a9-3355-4fbc-b7ea-fb88eb4e7c54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:33.288616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8466a9-3355-4fbc-b7ea-fb88eb4e7c54 None admin] RBAC: Adding query filter params (role.id=cb722313950945e5b147e994ec678cd1, scope.project.id=8c60d15f044842a0bc451a2e001075f0, user.id=fb74804ffc39429e9a10ef19845a532d) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:33.288918 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8466a9-3355-4fbc-b7ea-fb88eb4e7c54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:33.290356 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8466a9-3355-4fbc-b7ea-fb88eb4e7c54 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:33.302573 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 149/294] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:33 2022] GET /identity/v3/role_assignments?user.id=fb74804ffc39429e9a10ef19845a532d&scope.project.id=8c60d15f044842a0bc451a2e001075f0&role.id=cb722313950945e5b147e994ec678cd1 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:34.484400 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a8b3f-8e71-4ec3-8621-96bd7c31f75a None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:34.484963 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a8b3f-8e71-4ec3-8621-96bd7c31f75a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:34.485238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a8b3f-8e71-4ec3-8621-96bd7c31f75a None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:34.488157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 146/295] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:34.492877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742a846-b278-455e-ac64-305e6b84847a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:34.493806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742a846-b278-455e-ac64-305e6b84847a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:34.494274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742a846-b278-455e-ac64-305e6b84847a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:34.514366 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-9742a846-b278-455e-ac64-305e6b84847a None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:34.516537 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9742a846-b278-455e-ac64-305e6b84847a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:34.591602 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 150/296] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:34.619553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36011804-f997-4073-bcf8-6660da1d4cf3 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:34.620155 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36011804-f997-4073-bcf8-6660da1d4cf3 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:34.620625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36011804-f997-4073-bcf8-6660da1d4cf3 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:34.622417 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 147/297] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:34.627371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835df18f-08bc-4e7f-8eba-da4e71719771 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:34.627658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835df18f-08bc-4e7f-8eba-da4e71719771 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:34.627934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835df18f-08bc-4e7f-8eba-da4e71719771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:34.646988 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-835df18f-08bc-4e7f-8eba-da4e71719771 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:34.648186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-835df18f-08bc-4e7f-8eba-da4e71719771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:34.720016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 151/298] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:34.867975 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36011804-f997-4073-bcf8-6660da1d4cf3 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:34.975830 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36011804-f997-4073-bcf8-6660da1d4cf3 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.000839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:35.011726 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad43173-232d-4e4c-a695-6c8672ca2a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.012351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad43173-232d-4e4c-a695-6c8672ca2a5d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.012628 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad43173-232d-4e4c-a695-6c8672ca2a5d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.012896 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad43173-232d-4e4c-a695-6c8672ca2a5d None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.024452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad43173-232d-4e4c-a695-6c8672ca2a5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:35.029452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-6ad43173-232d-4e4c-a695-6c8672ca2a5d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:35.029452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:35.029452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:35.029452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:35.029452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:35.029452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:35.029452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:35.029452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:35.029452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:35.029452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:35.029452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:35.029452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:35.029452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:35.029452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:35.029452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:35.029452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:35.030840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:35.030840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:35.030840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:35.030840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:35.030840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:35.030840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:35.030840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:35.030840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:35.030840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:35.030840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:35.030840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:35.030840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:35.030840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:35.030840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:35.030840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:35.030840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:35.032255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:35.032255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:35.032255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:35.032255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:35.032255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:35.032255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:35.032255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:35.032255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:35.032255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:35.032255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:35.032255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:35.032255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:35.032255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 148/299] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:35.032911 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835df18f-08bc-4e7f-8eba-da4e71719771 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:35.041072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad43173-232d-4e4c-a695-6c8672ca2a5d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:35.071196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:35.164033 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad43173-232d-4e4c-a695-6c8672ca2a5d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.166662 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835df18f-08bc-4e7f-8eba-da4e71719771 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.207093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595f42b-8ef5-4cb6-a242-7b1760fe223a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.208360 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595f42b-8ef5-4cb6-a242-7b1760fe223a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.208817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595f42b-8ef5-4cb6-a242-7b1760fe223a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.209255 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595f42b-8ef5-4cb6-a242-7b1760fe223a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.211869 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2604b-9cff-4bd1-9978-9822d06485b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.212475 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2604b-9cff-4bd1-9978-9822d06485b3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.212753 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2604b-9cff-4bd1-9978-9822d06485b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.213026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2604b-9cff-4bd1-9978-9822d06485b3 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.214113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2604b-9cff-4bd1-9978-9822d06485b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:35.214417 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2604b-9cff-4bd1-9978-9822d06485b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:35.215837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2604b-9cff-4bd1-9978-9822d06485b3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:35.223208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3595f42b-8ef5-4cb6-a242-7b1760fe223a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:35.230334 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 149/300] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:35.231842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2604b-9cff-4bd1-9978-9822d06485b3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:35.233140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-3595f42b-8ef5-4cb6-a242-7b1760fe223a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:35.233140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:35.233140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:35.233140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:35.233140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:35.233140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:35.233140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:35.233140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:35.233140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:35.233140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:35.233140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:35.233140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:35.233140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:35.233140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:35.233140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:35.233140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:35.234539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:35.234539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:35.234539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:35.234539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:35.234539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:35.234539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:35.234539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:35.234539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:35.234539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:35.234539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:35.234539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:35.234539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:35.234539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:35.234539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:35.234539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:35.234539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:35.235374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:35.235374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:35.235374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:35.235374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:35.235374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:35.235374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:35.235374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:35.235374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:35.235374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:35.235374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:35.235374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:35.235374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:35.235374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 152/301] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:35.242312 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595f42b-8ef5-4cb6-a242-7b1760fe223a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:35.298476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:35.338374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:35.352193 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595f42b-8ef5-4cb6-a242-7b1760fe223a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.363414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2604b-9cff-4bd1-9978-9822d06485b3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.397933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fd5ce-e0bf-4303-985e-da6e5820f96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.398598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fd5ce-e0bf-4303-985e-da6e5820f96b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.398872 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fd5ce-e0bf-4303-985e-da6e5820f96b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.399177 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fd5ce-e0bf-4303-985e-da6e5820f96b None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.399816 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fd5ce-e0bf-4303-985e-da6e5820f96b None admin] RBAC: Adding query filter params (name=reader) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:35.400112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fd5ce-e0bf-4303-985e-da6e5820f96b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:35.401485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fd5ce-e0bf-4303-985e-da6e5820f96b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:35.403177 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf47e2-efbb-479f-ba7b-e14eb640e44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.403857 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf47e2-efbb-479f-ba7b-e14eb640e44e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.404081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf47e2-efbb-479f-ba7b-e14eb640e44e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.404345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf47e2-efbb-479f-ba7b-e14eb640e44e None admin] PATH_INFO: `/v3/users/7ced57aac59747a1b7ecdc96b1449539` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.406005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf47e2-efbb-479f-ba7b-e14eb640e44e None admin] RBAC: Authorizing `identity:get_user(user_id=7ced57aac59747a1b7ecdc96b1449539)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:35.407480 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bf47e2-efbb-479f-ba7b-e14eb640e44e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:35.409232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 150/302] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:35 2022] GET /identity/v3/users/7ced57aac59747a1b7ecdc96b1449539 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:35.410178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c33bdf-e92b-44fd-8c90-292063099384 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.410462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c33bdf-e92b-44fd-8c90-292063099384 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.410784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c33bdf-e92b-44fd-8c90-292063099384 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.412116 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 151/303] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:35.416330 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 153/304] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:35.416745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c33bdf-e92b-44fd-8c90-292063099384 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:35.417446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f62ba6-4112-4544-bf1e-03ad54594682 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.417732 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f62ba6-4112-4544-bf1e-03ad54594682 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.418067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f62ba6-4112-4544-bf1e-03ad54594682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.436298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b6f62ba6-4112-4544-bf1e-03ad54594682 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:35.437499 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b6f62ba6-4112-4544-bf1e-03ad54594682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:35.507123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 154/305] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:35.508638 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f62ba6-4112-4544-bf1e-03ad54594682 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:35.539720 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c33bdf-e92b-44fd-8c90-292063099384 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.554732 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:35.575320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a232529-6c9b-48b1-9680-306972e0e292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.576165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a232529-6c9b-48b1-9680-306972e0e292 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.576444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a232529-6c9b-48b1-9680-306972e0e292 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.576722 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a232529-6c9b-48b1-9680-306972e0e292 None admin] PATH_INFO: `/v3/system/users/7ced57aac59747a1b7ecdc96b1449539/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.579010 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a232529-6c9b-48b1-9680-306972e0e292 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7ced57aac59747a1b7ecdc96b1449539, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:35.580457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a232529-6c9b-48b1-9680-306972e0e292 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:35.594098 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:35.612192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f62ba6-4112-4544-bf1e-03ad54594682 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.645996 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8716b7-fa59-4be4-9715-8f5746b898b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.646615 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8716b7-fa59-4be4-9715-8f5746b898b5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.646901 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8716b7-fa59-4be4-9715-8f5746b898b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.647179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8716b7-fa59-4be4-9715-8f5746b898b5 None admin] PATH_INFO: `/v3/users/2f0bf6dbb8ad4c9db9d4e5c64d63d834` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.648377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8716b7-fa59-4be4-9715-8f5746b898b5 None admin] RBAC: Authorizing `identity:get_user(user_id=2f0bf6dbb8ad4c9db9d4e5c64d63d834)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:35.649802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8716b7-fa59-4be4-9715-8f5746b898b5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:35.651389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 155/306] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:35 2022] GET /identity/v3/users/2f0bf6dbb8ad4c9db9d4e5c64d63d834 => generated 297 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:35.652541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf665eb-d936-4c04-93a7-1493541e8e71 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.652822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf665eb-d936-4c04-93a7-1493541e8e71 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.653089 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf665eb-d936-4c04-93a7-1493541e8e71 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.654548 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 156/307] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:35.656788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf665eb-d936-4c04-93a7-1493541e8e71 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:35.683011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 152/308] 10.222.0.91 () {58 vars in 1334 bytes} [Thu Sep 8 13:37:35 2022] PUT /identity/v3/system/users/7ced57aac59747a1b7ecdc96b1449539/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:35.683011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b122c9-ea80-48bc-902e-e3fb83a0846f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.683011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b122c9-ea80-48bc-902e-e3fb83a0846f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.683011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b122c9-ea80-48bc-902e-e3fb83a0846f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.698228 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-52b122c9-ea80-48bc-902e-e3fb83a0846f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:35.700678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-52b122c9-ea80-48bc-902e-e3fb83a0846f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:35.790585 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf665eb-d936-4c04-93a7-1493541e8e71 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.918322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2604d72-df34-4ff8-92a3-d9020f9bdcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.927131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2604d72-df34-4ff8-92a3-d9020f9bdcc5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.927276 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 153/309] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:35.927370 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2604d72-df34-4ff8-92a3-d9020f9bdcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.927370 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2604d72-df34-4ff8-92a3-d9020f9bdcc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.927519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b122c9-ea80-48bc-902e-e3fb83a0846f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:35.950551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:36.004365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2604d72-df34-4ff8-92a3-d9020f9bdcc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:36.006009 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2604d72-df34-4ff8-92a3-d9020f9bdcc5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:36.052113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b122c9-ea80-48bc-902e-e3fb83a0846f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:36.065308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:36.097741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74f8f8-cef2-4222-9f5a-fdbdcdcefee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:36.098364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74f8f8-cef2-4222-9f5a-fdbdcdcefee1 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:36.098643 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74f8f8-cef2-4222-9f5a-fdbdcdcefee1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:36.098931 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74f8f8-cef2-4222-9f5a-fdbdcdcefee1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:36.113858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 157/310] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:37:36.115232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74f8f8-cef2-4222-9f5a-fdbdcdcefee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:36.119413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2604d72-df34-4ff8-92a3-d9020f9bdcc5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:36.120096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-3a74f8f8-cef2-4222-9f5a-fdbdcdcefee1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:36.120096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:36.120096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:36.120096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:36.120096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:36.120096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:36.120096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:36.120096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:36.120096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:36.120096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:36.120096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:36.120096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:36.120096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:36.120096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:36.120096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:36.120096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:36.120940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:36.120940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:36.120940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:36.120940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:36.120940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:36.120940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:36.120940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:36.120940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:36.120940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:36.120940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:36.120940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:36.120940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:36.120940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:36.120940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:36.120940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:36.120940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:36.122398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:36.122398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:36.122398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:36.122398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:36.122398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:36.122398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:36.122398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:36.122398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:36.122398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:36.122398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:36.122398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:36.122398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:36.122398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 154/311] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:36.128667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74f8f8-cef2-4222-9f5a-fdbdcdcefee1 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:36.191522 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:36.238184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2604d72-df34-4ff8-92a3-d9020f9bdcc5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:36.257229 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74f8f8-cef2-4222-9f5a-fdbdcdcefee1 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:36.274573 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadd492-a602-43e1-8981-bfbe96ef989b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:36.275286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadd492-a602-43e1-8981-bfbe96ef989b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:36.275561 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadd492-a602-43e1-8981-bfbe96ef989b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:36.275829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadd492-a602-43e1-8981-bfbe96ef989b None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:36.276935 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dadd492-a602-43e1-8981-bfbe96ef989b None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:36.278381 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dadd492-a602-43e1-8981-bfbe96ef989b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:36.280051 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 158/312] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:36 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:36.286111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadd492-a602-43e1-8981-bfbe96ef989b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:36.294031 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd212dfa-b857-487f-8397-00818b6b02fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:36.294633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd212dfa-b857-487f-8397-00818b6b02fc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:36.294934 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd212dfa-b857-487f-8397-00818b6b02fc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:36.295199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd212dfa-b857-487f-8397-00818b6b02fc None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:36.295846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd212dfa-b857-487f-8397-00818b6b02fc None admin] RBAC: Adding query filter params (name=reader) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:36.296141 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd212dfa-b857-487f-8397-00818b6b02fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:36.297497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd212dfa-b857-487f-8397-00818b6b02fc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:36.305779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 155/313] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:36.306372 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd212dfa-b857-487f-8397-00818b6b02fc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:36.339845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:36.434223 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadd492-a602-43e1-8981-bfbe96ef989b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:36.434664 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd212dfa-b857-487f-8397-00818b6b02fc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:36.451976 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:36.471797 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05198c8-dd56-4841-87bb-2cf7a5222501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:36.472089 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a30e2-554e-466f-954e-012daa2e117a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:36.472475 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05198c8-dd56-4841-87bb-2cf7a5222501 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:36.472671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a30e2-554e-466f-954e-012daa2e117a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:36.472768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05198c8-dd56-4841-87bb-2cf7a5222501 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:36.472998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a30e2-554e-466f-954e-012daa2e117a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:36.473091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05198c8-dd56-4841-87bb-2cf7a5222501 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:36.473266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a30e2-554e-466f-954e-012daa2e117a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:36.473999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a30e2-554e-466f-954e-012daa2e117a None admin] RBAC: Adding query filter params (role.id=d3fb1e3ec057480992710ad61b6cd1ed, scope.project.id=8c60d15f044842a0bc451a2e001075f0, user.id=2f0bf6dbb8ad4c9db9d4e5c64d63d834) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:36.474095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05198c8-dd56-4841-87bb-2cf7a5222501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:36.474306 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a30e2-554e-466f-954e-012daa2e117a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:36.478972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a30e2-554e-466f-954e-012daa2e117a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:36.479153 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05198c8-dd56-4841-87bb-2cf7a5222501 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:36.479153 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 159/314] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:36.479644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05198c8-dd56-4841-87bb-2cf7a5222501 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:36.489872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 156/315] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:36 2022] GET /identity/v3/role_assignments?user.id=2f0bf6dbb8ad4c9db9d4e5c64d63d834&scope.project.id=8c60d15f044842a0bc451a2e001075f0&role.id=d3fb1e3ec057480992710ad61b6cd1ed => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:36.491829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a30e2-554e-466f-954e-012daa2e117a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:36.618277 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:36.628260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a30e2-554e-466f-954e-012daa2e117a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:36.643584 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05198c8-dd56-4841-87bb-2cf7a5222501 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:36.676894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a91a0f-24bf-4e94-9780-24c3041b3e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:36.677869 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a91a0f-24bf-4e94-9780-24c3041b3e05 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:36.678364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a91a0f-24bf-4e94-9780-24c3041b3e05 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:36.678623 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a91a0f-24bf-4e94-9780-24c3041b3e05 None admin] PATH_INFO: `/v3/groups` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:36.679801 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a91a0f-24bf-4e94-9780-24c3041b3e05 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:36.681283 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a91a0f-24bf-4e94-9780-24c3041b3e05 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:36.684023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95eeb3-2f22-412c-9bda-8432b8d33628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:36.684649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95eeb3-2f22-412c-9bda-8432b8d33628 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:36.684923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95eeb3-2f22-412c-9bda-8432b8d33628 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:36.685194 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95eeb3-2f22-412c-9bda-8432b8d33628 None admin] PATH_INFO: `/v3/users/f1b81e4b472148c18b0bfbe2385f24ed` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:36.700088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95eeb3-2f22-412c-9bda-8432b8d33628 None admin] RBAC: Authorizing `identity:get_user(user_id=f1b81e4b472148c18b0bfbe2385f24ed)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:36.701638 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95eeb3-2f22-412c-9bda-8432b8d33628 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:36.704325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 160/316] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:36 2022] GET /identity/v3/users/f1b81e4b472148c18b0bfbe2385f24ed => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:36.709937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95eeb3-2f22-412c-9bda-8432b8d33628 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:36.757871 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 157/317] 10.222.0.91 () {60 vars in 1134 bytes} [Thu Sep 8 13:37:36 2022] POST /identity/v3/groups => generated 229 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:36.760685 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:36.835244 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95eeb3-2f22-412c-9bda-8432b8d33628 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:36.875073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49464bda-b2cb-4bda-9a2c-d20063f3fa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:36.875693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49464bda-b2cb-4bda-9a2c-d20063f3fa54 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:36.875973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49464bda-b2cb-4bda-9a2c-d20063f3fa54 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:36.876301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49464bda-b2cb-4bda-9a2c-d20063f3fa54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:36.877037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49464bda-b2cb-4bda-9a2c-d20063f3fa54 None admin] RBAC: Adding query filter params (role.id=d3fb1e3ec057480992710ad61b6cd1ed, user.id=f1b81e4b472148c18b0bfbe2385f24ed, scope.system=all) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:36.877346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49464bda-b2cb-4bda-9a2c-d20063f3fa54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:36.878828 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49464bda-b2cb-4bda-9a2c-d20063f3fa54 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:36.885367 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 161/318] 10.222.0.91 () {56 vars in 1308 bytes} [Thu Sep 8 13:37:36 2022] GET /identity/v3/role_assignments?user.id=f1b81e4b472148c18b0bfbe2385f24ed&scope.system=all&role.id=d3fb1e3ec057480992710ad61b6cd1ed => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:37.885178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4d518-7ee8-4102-ba0c-1e26ead1d02f None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:37.885786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4d518-7ee8-4102-ba0c-1e26ead1d02f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:37.885786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4d518-7ee8-4102-ba0c-1e26ead1d02f None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:37.887341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 158/319] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:37.893382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa81531-8449-43a0-906b-0a954657b535 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:37.893799 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa81531-8449-43a0-906b-0a954657b535 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:37.893938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa81531-8449-43a0-906b-0a954657b535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:37.914607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-8aa81531-8449-43a0-906b-0a954657b535 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:37.915895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8aa81531-8449-43a0-906b-0a954657b535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:37.986641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 162/320] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:38.253961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4d518-7ee8-4102-ba0c-1e26ead1d02f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:38.357086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4d518-7ee8-4102-ba0c-1e26ead1d02f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:38.392484 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9a45d-d3b8-4e5e-8575-b7627776ae24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:38.393107 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9a45d-d3b8-4e5e-8575-b7627776ae24 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.393383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9a45d-d3b8-4e5e-8575-b7627776ae24 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.393652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9a45d-d3b8-4e5e-8575-b7627776ae24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.404098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9a45d-d3b8-4e5e-8575-b7627776ae24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:38.410062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-d9a9a45d-d3b8-4e5e-8575-b7627776ae24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:38.410062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:38.410062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:38.410062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:38.410062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:38.410062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:38.410062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:38.410062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:38.410062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:38.410062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:38.410062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:38.410062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:38.410062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:38.410062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:38.410062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:38.410062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:38.410924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:38.410924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:38.410924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:38.410924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:38.410924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:38.410924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:38.410924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:38.410924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:38.410924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:38.410924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:38.410924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:38.410924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:38.410924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:38.410924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:38.410924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:38.410924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:38.411730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:38.411730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:38.411730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:38.411730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:38.411730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:38.411730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:38.411730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:38.411730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:38.411730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:38.411730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:38.411730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:38.411730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:38.415329 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 159/321] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:38.416941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa81531-8449-43a0-906b-0a954657b535 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:38.451787 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:38.522596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa81531-8449-43a0-906b-0a954657b535 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:38.557943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da990a-9448-4d8c-b78b-62df9d21ec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:38.558556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da990a-9448-4d8c-b78b-62df9d21ec04 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.558834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da990a-9448-4d8c-b78b-62df9d21ec04 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.559133 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da990a-9448-4d8c-b78b-62df9d21ec04 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.559790 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da990a-9448-4d8c-b78b-62df9d21ec04 None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:38.560090 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da990a-9448-4d8c-b78b-62df9d21ec04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:38.561518 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da990a-9448-4d8c-b78b-62df9d21ec04 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:38.570646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 163/322] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:38.576304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9a45d-d3b8-4e5e-8575-b7627776ae24 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:38.593885 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:38.677047 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9a45d-d3b8-4e5e-8575-b7627776ae24 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:38.712184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ff88a-9ccf-4efa-85d9-a3896d5e9d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:38.713016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ff88a-9ccf-4efa-85d9-a3896d5e9d91 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.713119 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ff88a-9ccf-4efa-85d9-a3896d5e9d91 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.713396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ff88a-9ccf-4efa-85d9-a3896d5e9d91 None admin] PATH_INFO: `/v3/users/7ced57aac59747a1b7ecdc96b1449539` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.714527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ff88a-9ccf-4efa-85d9-a3896d5e9d91 None admin] RBAC: Authorizing `identity:get_user(user_id=7ced57aac59747a1b7ecdc96b1449539)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:38.716009 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ff88a-9ccf-4efa-85d9-a3896d5e9d91 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:38.717740 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 160/323] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:38 2022] GET /identity/v3/users/7ced57aac59747a1b7ecdc96b1449539 => generated 293 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:38.723970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da990a-9448-4d8c-b78b-62df9d21ec04 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:38.820632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:38.825004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da990a-9448-4d8c-b78b-62df9d21ec04 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:38.863293 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305e9e3-25fd-4615-8c6b-5eff8dc3f6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:38.863920 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305e9e3-25fd-4615-8c6b-5eff8dc3f6fb None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.864198 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305e9e3-25fd-4615-8c6b-5eff8dc3f6fb None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.864470 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305e9e3-25fd-4615-8c6b-5eff8dc3f6fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.865213 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3305e9e3-25fd-4615-8c6b-5eff8dc3f6fb None admin] RBAC: Adding query filter params (role.id=cb722313950945e5b147e994ec678cd1, user.id=7ced57aac59747a1b7ecdc96b1449539, scope.system=all) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:38.865519 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3305e9e3-25fd-4615-8c6b-5eff8dc3f6fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:38.867057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3305e9e3-25fd-4615-8c6b-5eff8dc3f6fb None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:38.880868 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 164/324] 10.222.0.91 () {56 vars in 1308 bytes} [Thu Sep 8 13:37:38 2022] GET /identity/v3/role_assignments?user.id=7ced57aac59747a1b7ecdc96b1449539&scope.system=all&role.id=cb722313950945e5b147e994ec678cd1 => generated 510 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:38.976359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0a888-5de9-4504-8b2e-b7f38b659ce6 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.976359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0a888-5de9-4504-8b2e-b7f38b659ce6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.976970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0a888-5de9-4504-8b2e-b7f38b659ce6 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.978118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 161/325] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:38.984505 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1aa31d-5453-48fd-b681-77ba2efbbd7e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.985054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1aa31d-5453-48fd-b681-77ba2efbbd7e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.985054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1aa31d-5453-48fd-b681-77ba2efbbd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.004585 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1b1aa31d-5453-48fd-b681-77ba2efbbd7e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:39.005799 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1b1aa31d-5453-48fd-b681-77ba2efbbd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:39.076592 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba7aa6-f1a7-4e7b-b0d1-147d778f5d02 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.076783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba7aa6-f1a7-4e7b-b0d1-147d778f5d02 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.077066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ba7aa6-f1a7-4e7b-b0d1-147d778f5d02 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.079042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 162/326] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:39.081399 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 165/327] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:39.084042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585e6ce-d319-472c-b849-f872e399754d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.084324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585e6ce-d319-472c-b849-f872e399754d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.084601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585e6ce-d319-472c-b849-f872e399754d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.106097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f585e6ce-d319-472c-b849-f872e399754d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:39.107327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f585e6ce-d319-472c-b849-f872e399754d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:39.175458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 163/328] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:39.345398 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1aa31d-5453-48fd-b681-77ba2efbbd7e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:39.429637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:39.443857 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585e6ce-d319-472c-b849-f872e399754d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:39.450001 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1aa31d-5453-48fd-b681-77ba2efbbd7e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:39.458955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:39.483106 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf737322-7a0e-42fb-bfbd-d072251f2ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:39.483772 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf737322-7a0e-42fb-bfbd-d072251f2ddb None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.484047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf737322-7a0e-42fb-bfbd-d072251f2ddb None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.484313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf737322-7a0e-42fb-bfbd-d072251f2ddb None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.485402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf737322-7a0e-42fb-bfbd-d072251f2ddb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:39.486883 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf737322-7a0e-42fb-bfbd-d072251f2ddb None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:39.488472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 166/329] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:39.494155 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf737322-7a0e-42fb-bfbd-d072251f2ddb None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:39.549786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585e6ce-d319-472c-b849-f872e399754d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:39.583545 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8aa17-e42d-4c66-971d-ca569017d25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:39.584167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8aa17-e42d-4c66-971d-ca569017d25a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.584446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8aa17-e42d-4c66-971d-ca569017d25a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.584718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8aa17-e42d-4c66-971d-ca569017d25a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.592200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf737322-7a0e-42fb-bfbd-d072251f2ddb None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:39.594943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8aa17-e42d-4c66-971d-ca569017d25a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:39.599744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-eff8aa17-e42d-4c66-971d-ca569017d25a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:39.599744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:39.599744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:39.599744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:39.599744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:39.599744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:39.599744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:39.599744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:39.599744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:39.599744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:39.599744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:39.599744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:39.599744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:39.599744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:39.599744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:39.599744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:39.600672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:39.600672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:39.600672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:39.600672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:39.600672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:39.600672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:39.600672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:39.600672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:39.600672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:39.600672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:39.600672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:39.600672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:39.600672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:39.600672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:39.600672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:39.600672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:39.601729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:39.601729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:39.601729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:39.601729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:39.601729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:39.601729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:39.601729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:39.601729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:39.601729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:39.601729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:39.601729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:39.601729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:39.601729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 164/330] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:37:39.606498 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8aa17-e42d-4c66-971d-ca569017d25a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:39.627929 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d25b2-4472-4c9c-b0d2-2e3ca61e920a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:39.628543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d25b2-4472-4c9c-b0d2-2e3ca61e920a None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.628805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d25b2-4472-4c9c-b0d2-2e3ca61e920a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.629075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d25b2-4472-4c9c-b0d2-2e3ca61e920a None admin] PATH_INFO: `/v3/groups` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.629699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d25b2-4472-4c9c-b0d2-2e3ca61e920a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:39.631159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d25b2-4472-4c9c-b0d2-2e3ca61e920a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:39.676216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:39.704570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8aa17-e42d-4c66-971d-ca569017d25a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:39.719814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 167/331] 10.222.0.91 () {60 vars in 1134 bytes} [Thu Sep 8 13:37:39 2022] POST /identity/v3/groups => generated 226 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:37:39.742098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc10c65-396d-489a-af37-e6e61782d147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:39.742771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc10c65-396d-489a-af37-e6e61782d147 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.743421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc10c65-396d-489a-af37-e6e61782d147 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.743421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc10c65-396d-489a-af37-e6e61782d147 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.744002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc10c65-396d-489a-af37-e6e61782d147 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:39.744305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc10c65-396d-489a-af37-e6e61782d147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:39.745751 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc10c65-396d-489a-af37-e6e61782d147 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:39.756080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 165/332] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:39.761805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d25b2-4472-4c9c-b0d2-2e3ca61e920a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:39.779238 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:39.871877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d25b2-4472-4c9c-b0d2-2e3ca61e920a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:39.907163 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858cd8e-e73c-4b55-9131-c95ee4e3ca73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:39.907803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858cd8e-e73c-4b55-9131-c95ee4e3ca73 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.908084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858cd8e-e73c-4b55-9131-c95ee4e3ca73 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.908364 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858cd8e-e73c-4b55-9131-c95ee4e3ca73 None admin] PATH_INFO: `/v3/users/f1b81e4b472148c18b0bfbe2385f24ed` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.909494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8858cd8e-e73c-4b55-9131-c95ee4e3ca73 None admin] RBAC: Authorizing `identity:get_user(user_id=f1b81e4b472148c18b0bfbe2385f24ed)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:39.910991 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8858cd8e-e73c-4b55-9131-c95ee4e3ca73 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:39.912574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 168/333] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:39 2022] GET /identity/v3/users/f1b81e4b472148c18b0bfbe2385f24ed => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:39.918512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc10c65-396d-489a-af37-e6e61782d147 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:40.025180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc10c65-396d-489a-af37-e6e61782d147 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:40.064033 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e10bd-e56b-4a08-b8a0-a508afd7df3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:40.064682 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e10bd-e56b-4a08-b8a0-a508afd7df3e None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:40.064959 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e10bd-e56b-4a08-b8a0-a508afd7df3e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:40.065230 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e10bd-e56b-4a08-b8a0-a508afd7df3e None admin] PATH_INFO: `/v3/system/users/f1b81e4b472148c18b0bfbe2385f24ed/roles/d3fb1e3ec057480992710ad61b6cd1ed` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:40.066665 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e10bd-e56b-4a08-b8a0-a508afd7df3e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f1b81e4b472148c18b0bfbe2385f24ed, role_id=d3fb1e3ec057480992710ad61b6cd1ed)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:40.068183 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e10bd-e56b-4a08-b8a0-a508afd7df3e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:40.069928 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:40.130886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 166/334] 10.222.0.91 () {58 vars in 1334 bytes} [Thu Sep 8 13:37:39 2022] PUT /identity/v3/system/users/f1b81e4b472148c18b0bfbe2385f24ed/roles/d3fb1e3ec057480992710ad61b6cd1ed => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:42.101541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61b9ea-5f97-41de-a6f2-6d8a314b6ca1 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.102060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61b9ea-5f97-41de-a6f2-6d8a314b6ca1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.102060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61b9ea-5f97-41de-a6f2-6d8a314b6ca1 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.103813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 169/335] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:42.109428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29912a5b-c28a-4712-8451-2bbb6015e609 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.109872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29912a5b-c28a-4712-8451-2bbb6015e609 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.110046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29912a5b-c28a-4712-8451-2bbb6015e609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.132987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-29912a5b-c28a-4712-8451-2bbb6015e609 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:42.134201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-29912a5b-c28a-4712-8451-2bbb6015e609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:42.229560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 167/336] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:42.361264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8d445-7d25-4be2-9955-b186853788ab None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.361559 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8d445-7d25-4be2-9955-b186853788ab None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.361828 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8d445-7d25-4be2-9955-b186853788ab None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.363457 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 170/337] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:42.377978 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1350154-787d-4483-83a1-ff3d3dce0d3a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.378269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1350154-787d-4483-83a1-ff3d3dce0d3a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.378526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1350154-787d-4483-83a1-ff3d3dce0d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.407937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e1350154-787d-4483-83a1-ff3d3dce0d3a None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:42.412253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e1350154-787d-4483-83a1-ff3d3dce0d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:42.490923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 168/338] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:42.716723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8d445-7d25-4be2-9955-b186853788ab None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:42.821233 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:42.841127 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8d445-7d25-4be2-9955-b186853788ab None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:42.878687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5479ec2-c35d-4027-b142-cf37235173d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:42.879383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5479ec2-c35d-4027-b142-cf37235173d8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.879660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5479ec2-c35d-4027-b142-cf37235173d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.879926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5479ec2-c35d-4027-b142-cf37235173d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.883023 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e292c3-c6e3-4d4b-a164-0a0266f787a5 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.883458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e292c3-c6e3-4d4b-a164-0a0266f787a5 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.883921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e292c3-c6e3-4d4b-a164-0a0266f787a5 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.885554 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 169/339] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:42.891213 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24a632-193b-4fd0-bd6b-0ab4c9ef144f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.891493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24a632-193b-4fd0-bd6b-0ab4c9ef144f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.891756 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24a632-193b-4fd0-bd6b-0ab4c9ef144f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.901480 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5479ec2-c35d-4027-b142-cf37235173d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:42.906481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-b5479ec2-c35d-4027-b142-cf37235173d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:42.906481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:42.906481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:42.906481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:42.906481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:42.906481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:42.906481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:42.906481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:42.906481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:42.906481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:42.906481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:42.906481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:42.906481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:42.906481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:42.906481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:42.906481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:42.907436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:42.907436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:42.907436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:42.907436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:42.907436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:42.907436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:42.907436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:42.907436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:42.907436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:42.907436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:42.907436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:42.907436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:42.907436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:42.907436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:42.907436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:42.907436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:42.908234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:42.908234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:42.908234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:42.908234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:42.908234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:42.908234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:42.908234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:42.908234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:42.908234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:42.908234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:42.908234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:42.908234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:42.908234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 171/340] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:42.918636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5479ec2-c35d-4027-b142-cf37235173d8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:42.926877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ed24a632-193b-4fd0-bd6b-0ab4c9ef144f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:42.928191 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ed24a632-193b-4fd0-bd6b-0ab4c9ef144f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:43.019786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:43.036585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 170/341] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:43.038964 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24a632-193b-4fd0-bd6b-0ab4c9ef144f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:43.100228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5479ec2-c35d-4027-b142-cf37235173d8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:43.141441 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083735c-7a1e-47e4-bcb4-01c9b03515cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:43.142931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083735c-7a1e-47e4-bcb4-01c9b03515cd None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.143423 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083735c-7a1e-47e4-bcb4-01c9b03515cd None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.143917 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083735c-7a1e-47e4-bcb4-01c9b03515cd None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.144791 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083735c-7a1e-47e4-bcb4-01c9b03515cd None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:43.145307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083735c-7a1e-47e4-bcb4-01c9b03515cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:43.149016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083735c-7a1e-47e4-bcb4-01c9b03515cd None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:43.159058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 172/342] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:43.167263 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083735c-7a1e-47e4-bcb4-01c9b03515cd None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:43.168785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24a632-193b-4fd0-bd6b-0ab4c9ef144f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:43.173339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:43.218739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617c011-03b5-4adf-882e-8172875610a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:43.219430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617c011-03b5-4adf-882e-8172875610a0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.219711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617c011-03b5-4adf-882e-8172875610a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.220008 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617c011-03b5-4adf-882e-8172875610a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.237829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617c011-03b5-4adf-882e-8172875610a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:43.244741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-9617c011-03b5-4adf-882e-8172875610a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:43.244741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:43.244741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:43.244741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:43.244741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:43.244741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:43.244741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:43.244741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:43.244741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:43.244741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:43.244741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:43.244741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:43.244741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:43.244741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:43.244741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:43.244741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:43.245646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:43.245646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:43.245646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:43.245646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:43.245646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:43.245646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:43.245646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:43.245646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:43.245646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:43.245646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:43.245646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:43.245646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:43.245646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:43.245646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:43.245646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:43.245646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:43.246456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:43.246456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:43.246456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:43.246456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:43.246456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:43.246456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:43.246456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:43.246456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:43.246456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:43.246456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:43.246456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:43.246456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:43.246456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 171/343] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:43.252273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617c011-03b5-4adf-882e-8172875610a0 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:43.304839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083735c-7a1e-47e4-bcb4-01c9b03515cd None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:43.314201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:43.347441 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21e975-362d-46a9-8e4a-c4da22bc6805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:43.349105 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21e975-362d-46a9-8e4a-c4da22bc6805 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.349400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21e975-362d-46a9-8e4a-c4da22bc6805 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.349675 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21e975-362d-46a9-8e4a-c4da22bc6805 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.379130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617c011-03b5-4adf-882e-8172875610a0 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:43.423338 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50177f-46d9-4cf0-a5df-30c06efbcfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:43.424408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50177f-46d9-4cf0-a5df-30c06efbcfc7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.424894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50177f-46d9-4cf0-a5df-30c06efbcfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.425381 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50177f-46d9-4cf0-a5df-30c06efbcfc7 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.426531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50177f-46d9-4cf0-a5df-30c06efbcfc7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:43.427495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50177f-46d9-4cf0-a5df-30c06efbcfc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:43.429712 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50177f-46d9-4cf0-a5df-30c06efbcfc7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:43.438729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21e975-362d-46a9-8e4a-c4da22bc6805 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:43.440348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21e975-362d-46a9-8e4a-c4da22bc6805 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:43.441441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 172/344] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:43.444518 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:43.446781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50177f-46d9-4cf0-a5df-30c06efbcfc7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:43.567094 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 173/345] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:43.571125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50177f-46d9-4cf0-a5df-30c06efbcfc7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:43.575960 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21e975-362d-46a9-8e4a-c4da22bc6805 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:43.595866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:43.653665 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5316727-32e2-416d-8d16-4a89c3759f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:43.654590 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5316727-32e2-416d-8d16-4a89c3759f55 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.655088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5316727-32e2-416d-8d16-4a89c3759f55 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.655547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5316727-32e2-416d-8d16-4a89c3759f55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.776332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21e975-362d-46a9-8e4a-c4da22bc6805 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:43.777149 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5316727-32e2-416d-8d16-4a89c3759f55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:43.779058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:43.780079 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5316727-32e2-416d-8d16-4a89c3759f55 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:43.827068 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24076cc5-f8f6-4730-bea9-613dbe1c4fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:43.827068 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24076cc5-f8f6-4730-bea9-613dbe1c4fd8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.827068 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24076cc5-f8f6-4730-bea9-613dbe1c4fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.827649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24076cc5-f8f6-4730-bea9-613dbe1c4fd8 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.828988 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24076cc5-f8f6-4730-bea9-613dbe1c4fd8 None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:43.831936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24076cc5-f8f6-4730-bea9-613dbe1c4fd8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:43.834557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 174/346] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:43 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:43.835619 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24076cc5-f8f6-4730-bea9-613dbe1c4fd8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:43.916030 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 173/347] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:37:43.918530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a6af7-4977-4cff-85d4-bd0aeb45e91e None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.919056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a6af7-4977-4cff-85d4-bd0aeb45e91e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.919521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a6af7-4977-4cff-85d4-bd0aeb45e91e None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.922013 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 174/348] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:43.922926 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a6af7-4977-4cff-85d4-bd0aeb45e91e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:43.939212 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:43.977599 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24076cc5-f8f6-4730-bea9-613dbe1c4fd8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:44.019279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0ebbf-38a7-4599-8b4e-208cc0fa94ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:44.020446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0ebbf-38a7-4599-8b4e-208cc0fa94ca None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.020932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0ebbf-38a7-4599-8b4e-208cc0fa94ca None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.021408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0ebbf-38a7-4599-8b4e-208cc0fa94ca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.033643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0ebbf-38a7-4599-8b4e-208cc0fa94ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:44.039265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-a6b0ebbf-38a7-4599-8b4e-208cc0fa94ca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:44.039265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:44.039265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:44.039265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:44.039265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:44.039265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:44.039265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:44.039265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:44.039265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:44.039265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:44.039265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:44.039265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:44.039265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:44.039265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:44.039265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:44.039265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:44.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:44.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:44.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:44.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:44.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:44.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:44.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:44.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:44.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:44.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:44.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:44.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:44.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:44.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:44.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:44.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:44.040934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:44.040934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:44.040934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:44.040934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:44.040934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:44.040934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:44.040934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:44.040934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:44.040934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:44.040934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:44.040934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:44.040934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:44.042769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a6af7-4977-4cff-85d4-bd0aeb45e91e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:44.043042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 175/349] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:44.044077 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0ebbf-38a7-4599-8b4e-208cc0fa94ca None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:44.082663 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99f0f9-711b-4fae-912b-5d79a8a40881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:44.083057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:44.083767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99f0f9-711b-4fae-912b-5d79a8a40881 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.083767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99f0f9-711b-4fae-912b-5d79a8a40881 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.084285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99f0f9-711b-4fae-912b-5d79a8a40881 None admin] PATH_INFO: `/v3/groups/8593666e58f84301a8aa5e475a82089e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.091020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99f0f9-711b-4fae-912b-5d79a8a40881 None admin] RBAC: Authorizing `identity:get_group(group_id=8593666e58f84301a8aa5e475a82089e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:44.094361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f99f0f9-711b-4fae-912b-5d79a8a40881 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:44.095177 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 175/350] 10.222.0.91 () {56 vars in 1180 bytes} [Thu Sep 8 13:37:43 2022] GET /identity/v3/groups/8593666e58f84301a8aa5e475a82089e => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:44.096354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c47c4-c4e0-47b7-8af1-782b9d6a4cca None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.096638 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c47c4-c4e0-47b7-8af1-782b9d6a4cca None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.096913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c47c4-c4e0-47b7-8af1-782b9d6a4cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.117205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-df5c47c4-c4e0-47b7-8af1-782b9d6a4cca None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:44.118399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-df5c47c4-c4e0-47b7-8af1-782b9d6a4cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:44.172235 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0ebbf-38a7-4599-8b4e-208cc0fa94ca None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:44.196100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 176/351] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:44.196964 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c47c4-c4e0-47b7-8af1-782b9d6a4cca None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:44.205513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:44.223002 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90929ade-5ec1-49c8-8a6e-c703d5a0ca81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:44.224156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90929ade-5ec1-49c8-8a6e-c703d5a0ca81 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.224650 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90929ade-5ec1-49c8-8a6e-c703d5a0ca81 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.225137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90929ade-5ec1-49c8-8a6e-c703d5a0ca81 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.226511 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90929ade-5ec1-49c8-8a6e-c703d5a0ca81 None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:44.228191 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90929ade-5ec1-49c8-8a6e-c703d5a0ca81 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:44.230431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 176/352] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:44 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:44.231447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90929ade-5ec1-49c8-8a6e-c703d5a0ca81 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:44.304005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c47c4-c4e0-47b7-8af1-782b9d6a4cca None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:44.344983 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600513f9-a4c8-4fca-9a3b-42cfac3fba40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:44.345612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600513f9-a4c8-4fca-9a3b-42cfac3fba40 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.346012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600513f9-a4c8-4fca-9a3b-42cfac3fba40 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.346394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600513f9-a4c8-4fca-9a3b-42cfac3fba40 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.347214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600513f9-a4c8-4fca-9a3b-42cfac3fba40 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:44.347618 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600513f9-a4c8-4fca-9a3b-42cfac3fba40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:44.354974 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600513f9-a4c8-4fca-9a3b-42cfac3fba40 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:44.358262 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 177/353] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:44.359262 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600513f9-a4c8-4fca-9a3b-42cfac3fba40 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:44.384986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:44.388977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90929ade-5ec1-49c8-8a6e-c703d5a0ca81 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:44.429043 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c65ef-2fe0-4d45-b247-461249b9a33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:44.430065 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c65ef-2fe0-4d45-b247-461249b9a33d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.430827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:44.431644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c65ef-2fe0-4d45-b247-461249b9a33d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.432124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c65ef-2fe0-4d45-b247-461249b9a33d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.433104 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c65ef-2fe0-4d45-b247-461249b9a33d None admin] RBAC: Adding query filter params (group.id=8593666e58f84301a8aa5e475a82089e, role.id=cb722313950945e5b147e994ec678cd1, scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:44.433612 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c65ef-2fe0-4d45-b247-461249b9a33d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:44.435355 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c65ef-2fe0-4d45-b247-461249b9a33d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:44.442003 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 177/354] 10.222.0.91 () {56 vars in 1376 bytes} [Thu Sep 8 13:37:44 2022] GET /identity/v3/role_assignments?group.id=8593666e58f84301a8aa5e475a82089e&scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72&role.id=cb722313950945e5b147e994ec678cd1 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:44.443147 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c65ef-2fe0-4d45-b247-461249b9a33d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:44.472101 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600513f9-a4c8-4fca-9a3b-42cfac3fba40 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:44.512529 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eca7e5-dc93-4b7c-845b-de0cad4d0070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:44.513362 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eca7e5-dc93-4b7c-845b-de0cad4d0070 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.513644 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eca7e5-dc93-4b7c-845b-de0cad4d0070 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.513916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eca7e5-dc93-4b7c-845b-de0cad4d0070 None admin] PATH_INFO: `/v3/groups/8593666e58f84301a8aa5e475a82089e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.515607 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eca7e5-dc93-4b7c-845b-de0cad4d0070 None admin] RBAC: Authorizing `identity:get_group(group_id=8593666e58f84301a8aa5e475a82089e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:44.517173 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eca7e5-dc93-4b7c-845b-de0cad4d0070 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:44.519195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 178/355] 10.222.0.91 () {56 vars in 1180 bytes} [Thu Sep 8 13:37:44 2022] GET /identity/v3/groups/8593666e58f84301a8aa5e475a82089e => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:44.520650 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cfad4-cc38-4e57-8863-64c5ccd9f97b None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.521049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cfad4-cc38-4e57-8863-64c5ccd9f97b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.521219 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cfad4-cc38-4e57-8863-64c5ccd9f97b None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.523025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 179/356] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:44.524640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306cfad4-cc38-4e57-8863-64c5ccd9f97b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:44.580138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c65ef-2fe0-4d45-b247-461249b9a33d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:44.629859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbe464-6ce7-45cc-a369-927b50ff96d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:44.630761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbe464-6ce7-45cc-a369-927b50ff96d4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.631228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbe464-6ce7-45cc-a369-927b50ff96d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.631637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbe464-6ce7-45cc-a369-927b50ff96d4 None admin] PATH_INFO: `/v3/users/f1b81e4b472148c18b0bfbe2385f24ed` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.633426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbe464-6ce7-45cc-a369-927b50ff96d4 None admin] RBAC: Authorizing `identity:get_user(user_id=f1b81e4b472148c18b0bfbe2385f24ed)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:44.635166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbe464-6ce7-45cc-a369-927b50ff96d4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:44.637575 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 178/357] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:44 2022] GET /identity/v3/users/f1b81e4b472148c18b0bfbe2385f24ed => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:44.639099 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94138469-8f4c-423a-b2d9-970f9977bebd None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.639530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94138469-8f4c-423a-b2d9-970f9977bebd None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.640033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94138469-8f4c-423a-b2d9-970f9977bebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.649416 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306cfad4-cc38-4e57-8863-64c5ccd9f97b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:44.661417 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-94138469-8f4c-423a-b2d9-970f9977bebd None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:44.663124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-94138469-8f4c-423a-b2d9-970f9977bebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:44.689274 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf57b60-4d6e-4401-a4e8-226f02bcbf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:44.689892 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf57b60-4d6e-4401-a4e8-226f02bcbf18 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.690200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf57b60-4d6e-4401-a4e8-226f02bcbf18 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.690474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf57b60-4d6e-4401-a4e8-226f02bcbf18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.691244 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf57b60-4d6e-4401-a4e8-226f02bcbf18 None admin] RBAC: Adding query filter params (group.id=8593666e58f84301a8aa5e475a82089e, role.id=1a62bc555eb345fc8679cfa677069fd7, scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:44.691579 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf57b60-4d6e-4401-a4e8-226f02bcbf18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:44.693031 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf57b60-4d6e-4401-a4e8-226f02bcbf18 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:44.697832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:44.700624 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 180/358] 10.222.0.91 () {56 vars in 1376 bytes} [Thu Sep 8 13:37:44 2022] GET /identity/v3/role_assignments?group.id=8593666e58f84301a8aa5e475a82089e&scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72&role.id=1a62bc555eb345fc8679cfa677069fd7 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:44.701162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf57b60-4d6e-4401-a4e8-226f02bcbf18 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:44.713747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:44.767919 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 179/359] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:44.769328 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94138469-8f4c-423a-b2d9-970f9977bebd None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:44.820022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf57b60-4d6e-4401-a4e8-226f02bcbf18 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:44.860723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c01d34-efaa-40d8-846c-13b3f9e55e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:44.861361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c01d34-efaa-40d8-846c-13b3f9e55e25 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.861640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c01d34-efaa-40d8-846c-13b3f9e55e25 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.861902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c01d34-efaa-40d8-846c-13b3f9e55e25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.891400 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c01d34-efaa-40d8-846c-13b3f9e55e25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:44.897649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-f0c01d34-efaa-40d8-846c-13b3f9e55e25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:44.897649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:44.897649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:44.897649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:44.897649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:44.897649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:44.897649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:44.897649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:44.897649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:44.897649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:44.897649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:44.897649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:44.897649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:44.897649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:44.897649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:44.897649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:44.898671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:44.898671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:44.898671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:44.898671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:44.898671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:44.898671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:44.898671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:44.898671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:44.898671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:44.898671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:44.898671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:44.898671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:44.898671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:44.898671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:44.898671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:44.898671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:44.900932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:44.900932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:44.900932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:44.900932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:44.900932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:44.900932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:44.900932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:44.900932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:44.900932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:44.900932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:44.900932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:44.900932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:44.900932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 181/360] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:44.904532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94138469-8f4c-423a-b2d9-970f9977bebd None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:44.912386 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c01d34-efaa-40d8-846c-13b3f9e55e25 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:44.973223 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4d7f4-bdef-4517-88c7-f6685cf09f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:44.974246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4d7f4-bdef-4517-88c7-f6685cf09f55 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.974747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4d7f4-bdef-4517-88c7-f6685cf09f55 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.975342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4d7f4-bdef-4517-88c7-f6685cf09f55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.976467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4d7f4-bdef-4517-88c7-f6685cf09f55 None admin] RBAC: Adding query filter params (role.id=d3fb1e3ec057480992710ad61b6cd1ed, user.id=f1b81e4b472148c18b0bfbe2385f24ed, scope.system=all) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:44.977116 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4d7f4-bdef-4517-88c7-f6685cf09f55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:44.979524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4d7f4-bdef-4517-88c7-f6685cf09f55 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:44.998730 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 180/361] 10.222.0.91 () {56 vars in 1308 bytes} [Thu Sep 8 13:37:44 2022] GET /identity/v3/role_assignments?user.id=f1b81e4b472148c18b0bfbe2385f24ed&scope.system=all&role.id=d3fb1e3ec057480992710ad61b6cd1ed => generated 510 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:45.015945 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:45.065826 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c01d34-efaa-40d8-846c-13b3f9e55e25 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:45.104100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b76a0-9761-49ec-b8d9-d77d42f2cf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:45.105101 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b76a0-9761-49ec-b8d9-d77d42f2cf71 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:45.105581 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b76a0-9761-49ec-b8d9-d77d42f2cf71 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:45.106070 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b76a0-9761-49ec-b8d9-d77d42f2cf71 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:45.107022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b76a0-9761-49ec-b8d9-d77d42f2cf71 None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:45.107597 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b76a0-9761-49ec-b8d9-d77d42f2cf71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:45.109260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b76a0-9761-49ec-b8d9-d77d42f2cf71 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:45.119497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 182/362] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:45.128093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4d7f4-bdef-4517-88c7-f6685cf09f55 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:45.154311 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:45.253892 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4d7f4-bdef-4517-88c7-f6685cf09f55 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:45.263932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b76a0-9761-49ec-b8d9-d77d42f2cf71 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:45.308855 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af641b4-0d84-49f9-8c59-58ce53687b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:45.309472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af641b4-0d84-49f9-8c59-58ce53687b0f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:45.309749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af641b4-0d84-49f9-8c59-58ce53687b0f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:45.310028 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af641b4-0d84-49f9-8c59-58ce53687b0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:45.384078 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b76a0-9761-49ec-b8d9-d77d42f2cf71 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:45.393570 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af641b4-0d84-49f9-8c59-58ce53687b0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:45.395160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af641b4-0d84-49f9-8c59-58ce53687b0f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:45.406793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:45.422464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be7768-f273-4fd7-a747-eaca9953ce95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:45.426297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be7768-f273-4fd7-a747-eaca9953ce95 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:45.426297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be7768-f273-4fd7-a747-eaca9953ce95 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:45.426297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be7768-f273-4fd7-a747-eaca9953ce95 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:45.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:45.436924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be7768-f273-4fd7-a747-eaca9953ce95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:45.442510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-d5be7768-f273-4fd7-a747-eaca9953ce95 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:45.442510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:45.442510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:45.442510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:45.442510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:45.442510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:45.442510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:45.442510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:45.442510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:45.442510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:45.442510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:45.442510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:45.442510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:45.442510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:45.442510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:45.442510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:45.443495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:45.443495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:45.443495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:45.443495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:45.443495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:45.443495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:45.443495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:45.443495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:45.443495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:45.443495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:45.443495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:45.443495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:45.443495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:45.443495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:45.443495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:45.443495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:45.444343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:45.444343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:45.444343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:45.444343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:45.444343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:45.444343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:45.444343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:45.444343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:45.444343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:45.444343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:45.444343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:45.444343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:45.447377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 183/363] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:37:45.449074 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b6f99-47e0-4535-a373-bed0c3127f8a None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:45.449759 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b6f99-47e0-4535-a373-bed0c3127f8a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:45.450469 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b6f99-47e0-4535-a373-bed0c3127f8a None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:45.452836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 184/364] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:45.454018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b6f99-47e0-4535-a373-bed0c3127f8a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:45.582183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 181/365] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:45.582547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b6f99-47e0-4535-a373-bed0c3127f8a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:45.584367 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b6cc0-163d-4747-9e3b-9b3bb80f6e0e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:45.584894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b6cc0-163d-4747-9e3b-9b3bb80f6e0e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:45.586512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b6cc0-163d-4747-9e3b-9b3bb80f6e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:45.609549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c41b6cc0-163d-4747-9e3b-9b3bb80f6e0e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:45.612158 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c41b6cc0-163d-4747-9e3b-9b3bb80f6e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:45.626493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efdb04-b077-4f10-a42f-de583fb91728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:45.627171 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efdb04-b077-4f10-a42f-de583fb91728 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:45.627447 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efdb04-b077-4f10-a42f-de583fb91728 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:45.627717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efdb04-b077-4f10-a42f-de583fb91728 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:45.628379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efdb04-b077-4f10-a42f-de583fb91728 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:45.628681 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efdb04-b077-4f10-a42f-de583fb91728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:45.630194 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26efdb04-b077-4f10-a42f-de583fb91728 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:45.641169 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 185/366] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:45.644308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efdb04-b077-4f10-a42f-de583fb91728 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:45.696257 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:45.696425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 182/367] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:45.697515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b6cc0-163d-4747-9e3b-9b3bb80f6e0e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:45.761485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:45.764955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efdb04-b077-4f10-a42f-de583fb91728 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:45.802005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef9084-c705-4565-b32e-24ac8e3a1d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:45.803079 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef9084-c705-4565-b32e-24ac8e3a1d2b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:45.803982 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef9084-c705-4565-b32e-24ac8e3a1d2b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:45.804764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef9084-c705-4565-b32e-24ac8e3a1d2b None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:45.806184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef9084-c705-4565-b32e-24ac8e3a1d2b None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:45.807916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef9084-c705-4565-b32e-24ac8e3a1d2b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:45.809959 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 186/368] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:45 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:45.816383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef9084-c705-4565-b32e-24ac8e3a1d2b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:45.867268 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b6cc0-163d-4747-9e3b-9b3bb80f6e0e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:45.928986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16348d42-e20d-45d8-a37b-81026a866773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:45.929790 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16348d42-e20d-45d8-a37b-81026a866773 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:45.930016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16348d42-e20d-45d8-a37b-81026a866773 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:45.930290 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16348d42-e20d-45d8-a37b-81026a866773 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:45.966021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef9084-c705-4565-b32e-24ac8e3a1d2b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:45.978920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:46.007268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76977e-e556-48ee-b76b-bae92f781fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:46.008393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76977e-e556-48ee-b76b-bae92f781fe1 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:46.008873 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76977e-e556-48ee-b76b-bae92f781fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:46.009387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76977e-e556-48ee-b76b-bae92f781fe1 None admin] PATH_INFO: `/v3/groups/8593666e58f84301a8aa5e475a82089e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:46.010826 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76977e-e556-48ee-b76b-bae92f781fe1 None admin] RBAC: Authorizing `identity:get_group(group_id=8593666e58f84301a8aa5e475a82089e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:46.012522 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76977e-e556-48ee-b76b-bae92f781fe1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:46.014909 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 187/369] 10.222.0.91 () {56 vars in 1180 bytes} [Thu Sep 8 13:37:45 2022] GET /identity/v3/groups/8593666e58f84301a8aa5e475a82089e => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:46.020733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76977e-e556-48ee-b76b-bae92f781fe1 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:46.064648 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16348d42-e20d-45d8-a37b-81026a866773 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:46.066735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16348d42-e20d-45d8-a37b-81026a866773 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:46.168845 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:46.189214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76977e-e556-48ee-b76b-bae92f781fe1 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:46.229595 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 183/370] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:46.230485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16348d42-e20d-45d8-a37b-81026a866773 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:46.239002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9177f5-f5a2-4a26-b5c6-4d41c4c96d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:46.239713 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9177f5-f5a2-4a26-b5c6-4d41c4c96d23 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:46.239992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9177f5-f5a2-4a26-b5c6-4d41c4c96d23 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:46.240261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9177f5-f5a2-4a26-b5c6-4d41c4c96d23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:46.240977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9177f5-f5a2-4a26-b5c6-4d41c4c96d23 None admin] RBAC: Adding query filter params (group.id=8593666e58f84301a8aa5e475a82089e, role.id=cb722313950945e5b147e994ec678cd1, scope.project.id=8c60d15f044842a0bc451a2e001075f0) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:46.241577 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9177f5-f5a2-4a26-b5c6-4d41c4c96d23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:46.243205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9177f5-f5a2-4a26-b5c6-4d41c4c96d23 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:46.252807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 188/371] 10.222.0.91 () {56 vars in 1376 bytes} [Thu Sep 8 13:37:46 2022] GET /identity/v3/role_assignments?group.id=8593666e58f84301a8aa5e475a82089e&scope.project.id=8c60d15f044842a0bc451a2e001075f0&role.id=cb722313950945e5b147e994ec678cd1 => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:46.253682 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9177f5-f5a2-4a26-b5c6-4d41c4c96d23 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:46.338060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:46.434204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16348d42-e20d-45d8-a37b-81026a866773 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:46.450118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9177f5-f5a2-4a26-b5c6-4d41c4c96d23 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:46.508231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ccd7d-c253-4826-9971-1d68c9d762f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:46.509205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ccd7d-c253-4826-9971-1d68c9d762f2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:46.509484 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ccd7d-c253-4826-9971-1d68c9d762f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:46.509760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ccd7d-c253-4826-9971-1d68c9d762f2 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:46.511997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ccd7d-c253-4826-9971-1d68c9d762f2 None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:46.513483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ccd7d-c253-4826-9971-1d68c9d762f2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:46.515547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 189/372] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:46 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:46.521135 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ccd7d-c253-4826-9971-1d68c9d762f2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:46.523183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef34a4a-e11f-4937-b29c-eec92982c843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:46.524108 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef34a4a-e11f-4937-b29c-eec92982c843 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:46.524699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef34a4a-e11f-4937-b29c-eec92982c843 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:46.525174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef34a4a-e11f-4937-b29c-eec92982c843 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:46.549472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef34a4a-e11f-4937-b29c-eec92982c843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:46.556512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-3ef34a4a-e11f-4937-b29c-eec92982c843 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:46.556512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:46.556512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:46.556512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:46.556512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:46.556512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:46.556512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:46.556512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:46.556512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:46.556512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:46.556512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:46.556512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:46.556512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:46.556512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:46.556512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:46.556512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:46.557485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:46.557485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:46.557485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:46.557485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:46.557485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:46.557485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:46.557485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:46.557485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:46.557485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:46.557485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:46.557485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:46.557485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:46.557485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:46.557485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:46.557485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:46.557485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:46.558250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:46.558250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:46.558250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:46.558250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:46.558250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:46.558250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:46.558250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:46.558250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:46.558250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:46.558250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:46.558250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:46.558250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:46.560764 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 184/373] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:46.567643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef34a4a-e11f-4937-b29c-eec92982c843 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:46.588552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:46.687856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef34a4a-e11f-4937-b29c-eec92982c843 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:46.701803 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ccd7d-c253-4826-9971-1d68c9d762f2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:46.721670 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:46.725699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a6960-274b-4659-a81a-9a890608a313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:46.726456 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a6960-274b-4659-a81a-9a890608a313 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:46.726810 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a6960-274b-4659-a81a-9a890608a313 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:46.727238 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a6960-274b-4659-a81a-9a890608a313 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:46.728130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a6960-274b-4659-a81a-9a890608a313 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:46.728430 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a6960-274b-4659-a81a-9a890608a313 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:46.729815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a6960-274b-4659-a81a-9a890608a313 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:46.742082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 185/374] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:46.748787 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a6960-274b-4659-a81a-9a890608a313 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:46.769881 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5987310-9f27-42c2-97c6-9987e6429e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:46.770515 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5987310-9f27-42c2-97c6-9987e6429e2c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:46.770787 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5987310-9f27-42c2-97c6-9987e6429e2c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:46.771081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5987310-9f27-42c2-97c6-9987e6429e2c None admin] PATH_INFO: `/v3/groups/8593666e58f84301a8aa5e475a82089e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:46.772404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5987310-9f27-42c2-97c6-9987e6429e2c None admin] RBAC: Authorizing `identity:get_group(group_id=8593666e58f84301a8aa5e475a82089e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:46.773962 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5987310-9f27-42c2-97c6-9987e6429e2c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:46.779530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 190/375] 10.222.0.91 () {56 vars in 1180 bytes} [Thu Sep 8 13:37:46 2022] GET /identity/v3/groups/8593666e58f84301a8aa5e475a82089e => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:46.785780 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5987310-9f27-42c2-97c6-9987e6429e2c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:46.901643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:46.974155 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a6960-274b-4659-a81a-9a890608a313 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:46.978936 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5987310-9f27-42c2-97c6-9987e6429e2c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:47.028705 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9cd3e0-d789-4f80-98dd-c571f62d4586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:47.029252 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cd3e0-d789-4f80-98dd-c571f62d4586 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.029529 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cd3e0-d789-4f80-98dd-c571f62d4586 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.029799 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cd3e0-d789-4f80-98dd-c571f62d4586 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.030534 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9cd3e0-d789-4f80-98dd-c571f62d4586 None admin] RBAC: Adding query filter params (group.id=8593666e58f84301a8aa5e475a82089e, role.id=1a62bc555eb345fc8679cfa677069fd7, scope.project.id=8c60d15f044842a0bc451a2e001075f0) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:47.030878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9cd3e0-d789-4f80-98dd-c571f62d4586 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:47.032607 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9cd3e0-d789-4f80-98dd-c571f62d4586 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:47.038528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e42afc-838a-4460-a402-a856be38ccf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:47.039116 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 191/376] 10.222.0.91 () {56 vars in 1376 bytes} [Thu Sep 8 13:37:46 2022] GET /identity/v3/role_assignments?group.id=8593666e58f84301a8aa5e475a82089e&scope.project.id=8c60d15f044842a0bc451a2e001075f0&role.id=1a62bc555eb345fc8679cfa677069fd7 => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:47.039768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e42afc-838a-4460-a402-a856be38ccf8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.040275 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e42afc-838a-4460-a402-a856be38ccf8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.040747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e42afc-838a-4460-a402-a856be38ccf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.130451 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e42afc-838a-4460-a402-a856be38ccf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:47.132586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e42afc-838a-4460-a402-a856be38ccf8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:47.223210 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:47.262454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 186/377] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:47.269521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9cd3e0-d789-4f80-98dd-c571f62d4586 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:47.342098 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444f146-3cf0-4bae-91a3-fe8cbb91843a None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.342098 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444f146-3cf0-4bae-91a3-fe8cbb91843a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.342623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444f146-3cf0-4bae-91a3-fe8cbb91843a None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.344719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 187/378] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:47.347191 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:47.350491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd1f6e-9eef-4a0f-a700-c088516e024d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.352508 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd1f6e-9eef-4a0f-a700-c088516e024d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.352508 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd1f6e-9eef-4a0f-a700-c088516e024d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.383761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c0cd1f6e-9eef-4a0f-a700-c088516e024d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:47.385130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c0cd1f6e-9eef-4a0f-a700-c088516e024d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:47.422991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9cd3e0-d789-4f80-98dd-c571f62d4586 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:47.472308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818140b-94ed-4fb1-a498-3def9bd635f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:47.473189 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818140b-94ed-4fb1-a498-3def9bd635f4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.473634 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818140b-94ed-4fb1-a498-3def9bd635f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.474051 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818140b-94ed-4fb1-a498-3def9bd635f4 None admin] PATH_INFO: `/v3/projects/189571d729d54bff900838adb0030c5b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.491838 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818140b-94ed-4fb1-a498-3def9bd635f4 None admin] RBAC: Authorizing `identity:get_project(project_id=189571d729d54bff900838adb0030c5b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:47.493675 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818140b-94ed-4fb1-a498-3def9bd635f4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:47.495943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 192/379] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:47 2022] GET /identity/v3/projects/189571d729d54bff900838adb0030c5b => generated 344 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:47.502524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818140b-94ed-4fb1-a498-3def9bd635f4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:47.504825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 188/380] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:47.629795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818140b-94ed-4fb1-a498-3def9bd635f4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:47.641586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:47.676974 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1aff0-00e5-4301-a18e-cd32985c947d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:47.677882 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1aff0-00e5-4301-a18e-cd32985c947d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.678305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1aff0-00e5-4301-a18e-cd32985c947d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.678717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1aff0-00e5-4301-a18e-cd32985c947d None admin] PATH_INFO: `/v3/groups/1b4c2c5b07034085a5cb58d9bbf243f6` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.687699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1aff0-00e5-4301-a18e-cd32985c947d None admin] RBAC: Authorizing `identity:get_group(group_id=1b4c2c5b07034085a5cb58d9bbf243f6)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:47.689433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1aff0-00e5-4301-a18e-cd32985c947d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:47.691365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 193/381] 10.222.0.91 () {56 vars in 1180 bytes} [Thu Sep 8 13:37:47 2022] GET /identity/v3/groups/1b4c2c5b07034085a5cb58d9bbf243f6 => generated 229 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:47.697508 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd1f6e-9eef-4a0f-a700-c088516e024d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:47.823308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e042956-a622-46af-a975-ff55b559317f None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.823817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e042956-a622-46af-a975-ff55b559317f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.824234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e042956-a622-46af-a975-ff55b559317f None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.825726 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 194/382] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:47.831277 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a223ae-0349-4003-991f-563bc8826249 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.831561 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a223ae-0349-4003-991f-563bc8826249 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.831839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a223ae-0349-4003-991f-563bc8826249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.833378 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd1f6e-9eef-4a0f-a700-c088516e024d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:47.835181 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:47.854543 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-25a223ae-0349-4003-991f-563bc8826249 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:47.855941 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-25a223ae-0349-4003-991f-563bc8826249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:47.890582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027790a-1290-4e19-bd93-3c505479ceec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:47.891148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027790a-1290-4e19-bd93-3c505479ceec None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.891428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027790a-1290-4e19-bd93-3c505479ceec None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.891702 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027790a-1290-4e19-bd93-3c505479ceec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.892433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027790a-1290-4e19-bd93-3c505479ceec None admin] RBAC: Adding query filter params (group.id=1b4c2c5b07034085a5cb58d9bbf243f6, role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=189571d729d54bff900838adb0030c5b) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:47.892988 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027790a-1290-4e19-bd93-3c505479ceec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:47.895834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027790a-1290-4e19-bd93-3c505479ceec None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:47.908434 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 189/383] 10.222.0.91 () {56 vars in 1376 bytes} [Thu Sep 8 13:37:47 2022] GET /identity/v3/role_assignments?group.id=1b4c2c5b07034085a5cb58d9bbf243f6&scope.project.id=189571d729d54bff900838adb0030c5b&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:47.909644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027790a-1290-4e19-bd93-3c505479ceec None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:47.965923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 195/384] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:48.041658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027790a-1290-4e19-bd93-3c505479ceec None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:48.091295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69228a8-85d5-4245-8cf9-865a0945e929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:48.091949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69228a8-85d5-4245-8cf9-865a0945e929 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.097021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69228a8-85d5-4245-8cf9-865a0945e929 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.097312 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69228a8-85d5-4245-8cf9-865a0945e929 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.110478 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69228a8-85d5-4245-8cf9-865a0945e929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:48.115593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-e69228a8-85d5-4245-8cf9-865a0945e929 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:48.115593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:48.115593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:48.115593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:48.115593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:48.115593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:48.115593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:48.115593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:48.115593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:48.115593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:48.115593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:48.115593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:48.115593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:48.115593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:48.115593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:48.115593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:48.116540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:48.116540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:48.116540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:48.116540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:48.116540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:48.116540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:48.116540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:48.116540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:48.116540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:48.116540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:48.116540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:48.116540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:48.116540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:48.116540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:48.116540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:48.116540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:48.117338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:48.117338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:48.117338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:48.117338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:48.117338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:48.117338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:48.117338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:48.117338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:48.117338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:48.117338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:48.117338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:48.117338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:48.117338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 190/385] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:48.124019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a223ae-0349-4003-991f-563bc8826249 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:48.261371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a223ae-0349-4003-991f-563bc8826249 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:48.265022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:48.303245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000457c-4a27-4eeb-a279-423c79ac9133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:48.304531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000457c-4a27-4eeb-a279-423c79ac9133 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.305222 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000457c-4a27-4eeb-a279-423c79ac9133 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.305748 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000457c-4a27-4eeb-a279-423c79ac9133 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.306973 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000457c-4a27-4eeb-a279-423c79ac9133 None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:48.307286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000457c-4a27-4eeb-a279-423c79ac9133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:48.308688 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000457c-4a27-4eeb-a279-423c79ac9133 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:48.323637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 196/386] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:48.329913 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69228a8-85d5-4245-8cf9-865a0945e929 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:48.356056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000457c-4a27-4eeb-a279-423c79ac9133 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:48.381563 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:48.536494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69228a8-85d5-4245-8cf9-865a0945e929 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:48.564913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000457c-4a27-4eeb-a279-423c79ac9133 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:48.584426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3e2ca-c1e9-4404-bfa5-a3e6d4441c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:48.585075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3e2ca-c1e9-4404-bfa5-a3e6d4441c2f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.585344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3e2ca-c1e9-4404-bfa5-a3e6d4441c2f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.585681 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3e2ca-c1e9-4404-bfa5-a3e6d4441c2f None admin] PATH_INFO: `/v3/groups/8593666e58f84301a8aa5e475a82089e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.586853 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3e2ca-c1e9-4404-bfa5-a3e6d4441c2f None admin] RBAC: Authorizing `identity:get_group(group_id=8593666e58f84301a8aa5e475a82089e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:48.588462 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3e2ca-c1e9-4404-bfa5-a3e6d4441c2f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:48.590128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 191/387] 10.222.0.91 () {56 vars in 1180 bytes} [Thu Sep 8 13:37:48 2022] GET /identity/v3/groups/8593666e58f84301a8aa5e475a82089e => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:48.596700 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3e2ca-c1e9-4404-bfa5-a3e6d4441c2f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:48.672911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334f5e4-cd09-4d4d-8fee-4513629e4104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:48.673561 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334f5e4-cd09-4d4d-8fee-4513629e4104 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.673832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334f5e4-cd09-4d4d-8fee-4513629e4104 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.674100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334f5e4-cd09-4d4d-8fee-4513629e4104 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.711532 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334f5e4-cd09-4d4d-8fee-4513629e4104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:48.715295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:48.716868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-0334f5e4-cd09-4d4d-8fee-4513629e4104 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:48.716868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:48.716868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:48.716868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:48.716868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:48.716868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:48.716868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:48.716868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:48.716868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:48.716868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:48.716868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:48.716868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:48.716868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:48.716868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:48.716868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:48.716868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:48.717723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:48.717723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:48.717723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:48.717723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:48.717723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:48.717723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:48.717723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:48.717723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:48.717723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:48.717723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:48.717723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:48.717723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:48.717723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:48.717723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:48.717723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:48.717723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:48.719106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:48.719106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:48.719106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:48.719106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:48.719106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:48.719106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:48.719106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:48.719106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:48.719106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:48.719106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:48.719106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:48.719106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:48.719106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 197/388] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:37:48.721919 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3e2ca-c1e9-4404-bfa5-a3e6d4441c2f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:48.734860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334f5e4-cd09-4d4d-8fee-4513629e4104 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:48.765182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b902366-6589-415c-ac02-8149772ef429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:48.766631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b902366-6589-415c-ac02-8149772ef429 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.767096 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b902366-6589-415c-ac02-8149772ef429 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.767535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b902366-6589-415c-ac02-8149772ef429 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.769326 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b902366-6589-415c-ac02-8149772ef429 None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:48.771247 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b902366-6589-415c-ac02-8149772ef429 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:48.773809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 192/389] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:48 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:48.779333 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b902366-6589-415c-ac02-8149772ef429 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:48.831161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:48.906332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b902366-6589-415c-ac02-8149772ef429 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:48.949038 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3fbd6-c3b6-4c61-98a0-6e026211caf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:48.950003 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3fbd6-c3b6-4c61-98a0-6e026211caf1 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.950421 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3fbd6-c3b6-4c61-98a0-6e026211caf1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.951030 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3fbd6-c3b6-4c61-98a0-6e026211caf1 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72/groups/8593666e58f84301a8aa5e475a82089e/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.954686 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3fbd6-c3b6-4c61-98a0-6e026211caf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fc04ff6a9f94d04af0a564c8a1bab72, group_id=8593666e58f84301a8aa5e475a82089e, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:48.956453 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3fbd6-c3b6-4c61-98a0-6e026211caf1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:49.004301 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334f5e4-cd09-4d4d-8fee-4513629e4104 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:49.044769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3adc69-c619-404c-8e4a-34e6640d9752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:49.053006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 193/390] 10.222.0.91 () {58 vars in 1442 bytes} [Thu Sep 8 13:37:48 2022] PUT /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72/groups/8593666e58f84301a8aa5e475a82089e/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:49.054121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3adc69-c619-404c-8e4a-34e6640d9752 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:49.054607 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3adc69-c619-404c-8e4a-34e6640d9752 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:49.055109 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3adc69-c619-404c-8e4a-34e6640d9752 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:49.056167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3adc69-c619-404c-8e4a-34e6640d9752 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:49.056637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3adc69-c619-404c-8e4a-34e6640d9752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:49.058412 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3adc69-c619-404c-8e4a-34e6640d9752 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:49.070253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 198/391] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:49.076526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3fbd6-c3b6-4c61-98a0-6e026211caf1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:49.107054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:49.205547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3fbd6-c3b6-4c61-98a0-6e026211caf1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:49.253645 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b693b8a-b4c1-4799-8ceb-85873f7d2dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:49.254629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b693b8a-b4c1-4799-8ceb-85873f7d2dfb None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:49.255095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b693b8a-b4c1-4799-8ceb-85873f7d2dfb None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:49.255573 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b693b8a-b4c1-4799-8ceb-85873f7d2dfb None admin] PATH_INFO: `/v3/groups/8593666e58f84301a8aa5e475a82089e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:49.257088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b693b8a-b4c1-4799-8ceb-85873f7d2dfb None admin] RBAC: Authorizing `identity:get_group(group_id=8593666e58f84301a8aa5e475a82089e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:49.258838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b693b8a-b4c1-4799-8ceb-85873f7d2dfb None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:49.260977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 194/392] 10.222.0.91 () {56 vars in 1180 bytes} [Thu Sep 8 13:37:49 2022] GET /identity/v3/groups/8593666e58f84301a8aa5e475a82089e => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:49.267457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3adc69-c619-404c-8e4a-34e6640d9752 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:49.353002 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4273c-c3c9-4715-a4ba-a10536c4e88f None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:49.353395 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4273c-c3c9-4715-a4ba-a10536c4e88f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:49.353787 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4273c-c3c9-4715-a4ba-a10536c4e88f None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:49.355658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 195/393] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:49.364761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91142efc-3b3c-42e1-8932-ad56f7dab396 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:49.364761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91142efc-3b3c-42e1-8932-ad56f7dab396 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:49.365274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91142efc-3b3c-42e1-8932-ad56f7dab396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:49.378551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:49.395447 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3adc69-c619-404c-8e4a-34e6640d9752 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:49.399784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-91142efc-3b3c-42e1-8932-ad56f7dab396 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:49.401235 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-91142efc-3b3c-42e1-8932-ad56f7dab396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:49.435154 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d04774-2cbd-4a9b-b46b-31909c927391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:49.436277 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d04774-2cbd-4a9b-b46b-31909c927391 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:49.437687 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d04774-2cbd-4a9b-b46b-31909c927391 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:49.438128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d04774-2cbd-4a9b-b46b-31909c927391 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:49.439897 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d04774-2cbd-4a9b-b46b-31909c927391 None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:49.441572 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d04774-2cbd-4a9b-b46b-31909c927391 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:49.443781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 199/394] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:49 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:49.450110 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d04774-2cbd-4a9b-b46b-31909c927391 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:49.494742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 196/395] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:49.566016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d04774-2cbd-4a9b-b46b-31909c927391 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:49.603516 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f337e0-7c68-4880-a998-63a3b3c1d368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:49.604568 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f337e0-7c68-4880-a998-63a3b3c1d368 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:49.605029 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f337e0-7c68-4880-a998-63a3b3c1d368 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:49.605533 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f337e0-7c68-4880-a998-63a3b3c1d368 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72/groups/8593666e58f84301a8aa5e475a82089e/roles/1a62bc555eb345fc8679cfa677069fd7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:49.608438 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f337e0-7c68-4880-a998-63a3b3c1d368 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fc04ff6a9f94d04af0a564c8a1bab72, group_id=8593666e58f84301a8aa5e475a82089e, role_id=1a62bc555eb345fc8679cfa677069fd7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:49.610800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f337e0-7c68-4880-a998-63a3b3c1d368 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:49.693335 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 200/396] 10.222.0.91 () {58 vars in 1442 bytes} [Thu Sep 8 13:37:49 2022] PUT /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72/groups/8593666e58f84301a8aa5e475a82089e/roles/1a62bc555eb345fc8679cfa677069fd7 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:37:49.843312 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91142efc-3b3c-42e1-8932-ad56f7dab396 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:49.868181 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:49.958802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91142efc-3b3c-42e1-8932-ad56f7dab396 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:49.998391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b7615-2c3a-4410-87ee-e554ac4b90b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:49.999087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b7615-2c3a-4410-87ee-e554ac4b90b4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:49.999366 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b7615-2c3a-4410-87ee-e554ac4b90b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:49.999636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b7615-2c3a-4410-87ee-e554ac4b90b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:50.012950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b7615-2c3a-4410-87ee-e554ac4b90b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:50.018535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-432b7615-2c3a-4410-87ee-e554ac4b90b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:50.018535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:50.018535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:50.018535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:50.018535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:50.018535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:50.018535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:50.018535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:50.018535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:50.018535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:50.018535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:50.018535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:50.018535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:50.018535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:50.018535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:50.018535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:50.019515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:50.019515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:50.019515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:50.019515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:50.019515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:50.019515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:50.019515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:50.019515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:50.019515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:50.019515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:50.019515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:50.019515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:50.019515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:50.019515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:50.019515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:50.019515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:50.020303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:50.020303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:50.020303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:50.020303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:50.020303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:50.020303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:50.020303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:50.020303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:50.020303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:50.020303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:50.020303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:50.020303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:50.020303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 197/397] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:50.026400 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f337e0-7c68-4880-a998-63a3b3c1d368 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:50.054530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:50.168355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f337e0-7c68-4880-a998-63a3b3c1d368 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:50.210225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c7e9c-72de-4e85-91de-43f0686f7322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:50.210774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c7e9c-72de-4e85-91de-43f0686f7322 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:50.211085 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c7e9c-72de-4e85-91de-43f0686f7322 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:50.211362 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c7e9c-72de-4e85-91de-43f0686f7322 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:50.212044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c7e9c-72de-4e85-91de-43f0686f7322 None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:50.212349 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c7e9c-72de-4e85-91de-43f0686f7322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:50.213743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c7e9c-72de-4e85-91de-43f0686f7322 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:50.227020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 201/398] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:50.232090 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b7615-2c3a-4410-87ee-e554ac4b90b4 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:50.276286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c459d3-a88b-4313-8d5a-5149319aaf59 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:50.276603 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c459d3-a88b-4313-8d5a-5149319aaf59 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:50.276848 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c459d3-a88b-4313-8d5a-5149319aaf59 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:50.278500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 202/399] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:50.284271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554775d-ffe1-463d-af3c-800419dbfece None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:50.284565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554775d-ffe1-463d-af3c-800419dbfece None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:50.284816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554775d-ffe1-463d-af3c-800419dbfece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:50.317360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4554775d-ffe1-463d-af3c-800419dbfece None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:50.318576 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4554775d-ffe1-463d-af3c-800419dbfece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:50.344619 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:50.383561 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b7615-2c3a-4410-87ee-e554ac4b90b4 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:50.428138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11bea6-2a04-4bed-acc7-ed14530113fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:50.428793 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11bea6-2a04-4bed-acc7-ed14530113fc None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:50.429108 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11bea6-2a04-4bed-acc7-ed14530113fc None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:50.429395 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11bea6-2a04-4bed-acc7-ed14530113fc None admin] PATH_INFO: `/v3/groups/8593666e58f84301a8aa5e475a82089e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:50.431163 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11bea6-2a04-4bed-acc7-ed14530113fc None admin] RBAC: Authorizing `identity:get_group(group_id=8593666e58f84301a8aa5e475a82089e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:50.432635 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11bea6-2a04-4bed-acc7-ed14530113fc None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:50.438920 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 198/400] 10.222.0.91 () {56 vars in 1180 bytes} [Thu Sep 8 13:37:50 2022] GET /identity/v3/groups/8593666e58f84301a8aa5e475a82089e => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:50.440967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11bea6-2a04-4bed-acc7-ed14530113fc None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:50.463304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 203/401] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:50.559575 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11bea6-2a04-4bed-acc7-ed14530113fc None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:50.599859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5436427-7b87-4d14-9f00-17e90abfef06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:50.600501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5436427-7b87-4d14-9f00-17e90abfef06 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:50.600776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5436427-7b87-4d14-9f00-17e90abfef06 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:50.601047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5436427-7b87-4d14-9f00-17e90abfef06 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:50.602345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5436427-7b87-4d14-9f00-17e90abfef06 None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:50.603860 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5436427-7b87-4d14-9f00-17e90abfef06 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:50.605865 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 199/402] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:50 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:50.615991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554775d-ffe1-463d-af3c-800419dbfece None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:50.633348 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:50.766969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554775d-ffe1-463d-af3c-800419dbfece None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:50.814125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e90135-a8fe-4ac5-b452-e08597bb5465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:50.815185 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e90135-a8fe-4ac5-b452-e08597bb5465 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:50.815665 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e90135-a8fe-4ac5-b452-e08597bb5465 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:50.816156 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e90135-a8fe-4ac5-b452-e08597bb5465 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0/groups/8593666e58f84301a8aa5e475a82089e/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:50.824370 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e90135-a8fe-4ac5-b452-e08597bb5465 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c60d15f044842a0bc451a2e001075f0, group_id=8593666e58f84301a8aa5e475a82089e, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:50.826136 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e90135-a8fe-4ac5-b452-e08597bb5465 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:50.842227 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26483861-b5ea-41bd-b97b-539f63f61539 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:50.842507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26483861-b5ea-41bd-b97b-539f63f61539 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:50.857777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26483861-b5ea-41bd-b97b-539f63f61539 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:50.857777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 200/403] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:50.857777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b212644-5370-4cd3-b93f-0aa2e4c67a16 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:50.857777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b212644-5370-4cd3-b93f-0aa2e4c67a16 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:50.857777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b212644-5370-4cd3-b93f-0aa2e4c67a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:50.877714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-0b212644-5370-4cd3-b93f-0aa2e4c67a16 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:50.878977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0b212644-5370-4cd3-b93f-0aa2e4c67a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:50.895563 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:50.968631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 201/404] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:50.969662 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b212644-5370-4cd3-b93f-0aa2e4c67a16 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:50.990365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 204/405] 10.222.0.91 () {58 vars in 1442 bytes} [Thu Sep 8 13:37:50 2022] PUT /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0/groups/8593666e58f84301a8aa5e475a82089e/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:51.117641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b212644-5370-4cd3-b93f-0aa2e4c67a16 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:51.168313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc2ab9-bbea-4afe-9fdd-1864434eec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:51.169003 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc2ab9-bbea-4afe-9fdd-1864434eec65 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:51.169290 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc2ab9-bbea-4afe-9fdd-1864434eec65 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:51.169594 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc2ab9-bbea-4afe-9fdd-1864434eec65 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:51.185523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc2ab9-bbea-4afe-9fdd-1864434eec65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:51.191440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-48cc2ab9-bbea-4afe-9fdd-1864434eec65 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:51.191440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:51.191440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:51.191440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:51.191440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:51.191440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:51.191440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:51.191440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:51.191440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:51.191440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:51.191440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:51.191440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:51.191440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:51.191440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:51.191440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:51.191440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:51.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:51.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:51.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:51.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:51.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:51.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:51.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:51.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:51.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:51.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:51.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:51.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:51.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:51.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:51.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:51.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:51.193166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:51.193166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:51.193166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:51.193166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:51.193166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:51.193166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:51.193166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:51.193166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:51.193166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:51.193166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:51.193166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:51.193166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:51.193166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 202/406] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:51.198870 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e90135-a8fe-4ac5-b452-e08597bb5465 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:51.300408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:51.314151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc2ab9-bbea-4afe-9fdd-1864434eec65 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:51.337638 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:51.346018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e90135-a8fe-4ac5-b452-e08597bb5465 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:51.385916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a04074-ad7a-4d94-8d8f-bd91f5e25d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:51.386451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a04074-ad7a-4d94-8d8f-bd91f5e25d01 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:51.386732 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a04074-ad7a-4d94-8d8f-bd91f5e25d01 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:51.387124 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a04074-ad7a-4d94-8d8f-bd91f5e25d01 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:51.387806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a04074-ad7a-4d94-8d8f-bd91f5e25d01 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:51.388105 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a04074-ad7a-4d94-8d8f-bd91f5e25d01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:51.389501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a04074-ad7a-4d94-8d8f-bd91f5e25d01 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:51.399085 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 205/407] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:51.406934 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a04074-ad7a-4d94-8d8f-bd91f5e25d01 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:51.499830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc2ab9-bbea-4afe-9fdd-1864434eec65 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:51.580204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed33c4e-fe77-416c-a86f-b4963cc176d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:51.581235 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed33c4e-fe77-416c-a86f-b4963cc176d8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:51.581891 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed33c4e-fe77-416c-a86f-b4963cc176d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:51.582376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed33c4e-fe77-416c-a86f-b4963cc176d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:51.585029 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a04074-ad7a-4d94-8d8f-bd91f5e25d01 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:51.604651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed33c4e-fe77-416c-a86f-b4963cc176d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:51.617517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-9ed33c4e-fe77-416c-a86f-b4963cc176d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:51.617517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:51.617517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:51.617517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:51.617517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:51.617517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:51.617517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:51.617517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:51.617517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:51.617517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:51.617517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:51.617517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:51.617517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:51.617517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:51.617517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:51.617517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:51.618427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:51.618427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:51.618427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:51.618427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:51.618427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:51.618427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:51.618427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:51.618427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:51.618427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:51.618427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:51.618427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:51.618427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:51.618427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:51.618427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:51.618427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:51.618427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:51.619242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:51.619242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:51.619242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:51.619242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:51.619242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:51.619242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:51.619242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:51.619242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:51.619242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:51.619242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:51.619242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:51.619242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:37:51.621882 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 203/408] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:51.623331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:51.628636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed33c4e-fe77-416c-a86f-b4963cc176d8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:51.635761 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800ab5c-508e-4158-a184-38891f5a4af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:51.652837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800ab5c-508e-4158-a184-38891f5a4af9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:51.652837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800ab5c-508e-4158-a184-38891f5a4af9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:51.652837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800ab5c-508e-4158-a184-38891f5a4af9 None admin] PATH_INFO: `/v3/groups/8593666e58f84301a8aa5e475a82089e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:51.652837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800ab5c-508e-4158-a184-38891f5a4af9 None admin] RBAC: Authorizing `identity:get_group(group_id=8593666e58f84301a8aa5e475a82089e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:51.652837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800ab5c-508e-4158-a184-38891f5a4af9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:51.652837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 206/409] 10.222.0.91 () {56 vars in 1180 bytes} [Thu Sep 8 13:37:51 2022] GET /identity/v3/groups/8593666e58f84301a8aa5e475a82089e => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:51.653383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800ab5c-508e-4158-a184-38891f5a4af9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:51.705171 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:51.747036 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed33c4e-fe77-416c-a86f-b4963cc176d8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:51.818647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92886b-80e2-481c-b457-d24a522661e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:51.819844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92886b-80e2-481c-b457-d24a522661e5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:51.820350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92886b-80e2-481c-b457-d24a522661e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:51.820858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92886b-80e2-481c-b457-d24a522661e5 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:51.821783 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92886b-80e2-481c-b457-d24a522661e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:51.822342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92886b-80e2-481c-b457-d24a522661e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:51.824081 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92886b-80e2-481c-b457-d24a522661e5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:51.839100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 204/410] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:51.846600 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92886b-80e2-481c-b457-d24a522661e5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:51.849075 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800ab5c-508e-4158-a184-38891f5a4af9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:51.927848 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4b716-9c5a-467c-854e-4d578132087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:51.928857 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4b716-9c5a-467c-854e-4d578132087f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:51.929329 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4b716-9c5a-467c-854e-4d578132087f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:51.929790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4b716-9c5a-467c-854e-4d578132087f None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:51.932038 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4b716-9c5a-467c-854e-4d578132087f None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:51.933746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4b716-9c5a-467c-854e-4d578132087f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:51.936503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 207/411] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:51 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:51.942423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4b716-9c5a-467c-854e-4d578132087f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:52.042584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:52.056033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92886b-80e2-481c-b457-d24a522661e5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:52.107189 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4b716-9c5a-467c-854e-4d578132087f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:52.112017 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:52.132789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15df83f-a508-4a4e-9343-c23bc5bece9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:52.133468 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15df83f-a508-4a4e-9343-c23bc5bece9d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:52.133751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15df83f-a508-4a4e-9343-c23bc5bece9d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:52.134020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15df83f-a508-4a4e-9343-c23bc5bece9d None admin] PATH_INFO: `/v3/groups/1b4c2c5b07034085a5cb58d9bbf243f6` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:52.136423 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15df83f-a508-4a4e-9343-c23bc5bece9d None admin] RBAC: Authorizing `identity:get_group(group_id=1b4c2c5b07034085a5cb58d9bbf243f6)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:52.137928 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15df83f-a508-4a4e-9343-c23bc5bece9d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:52.140178 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 205/412] 10.222.0.91 () {56 vars in 1180 bytes} [Thu Sep 8 13:37:51 2022] GET /identity/v3/groups/1b4c2c5b07034085a5cb58d9bbf243f6 => generated 229 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:52.147593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15df83f-a508-4a4e-9343-c23bc5bece9d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:52.190513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9b24b-d604-445a-85ca-6fa747daf9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:52.191140 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9b24b-d604-445a-85ca-6fa747daf9e2 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:52.191415 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9b24b-d604-445a-85ca-6fa747daf9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:52.191686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9b24b-d604-445a-85ca-6fa747daf9e2 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0/groups/8593666e58f84301a8aa5e475a82089e/roles/1a62bc555eb345fc8679cfa677069fd7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:52.195189 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9b24b-d604-445a-85ca-6fa747daf9e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8c60d15f044842a0bc451a2e001075f0, group_id=8593666e58f84301a8aa5e475a82089e, role_id=1a62bc555eb345fc8679cfa677069fd7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:52.196651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9b24b-d604-445a-85ca-6fa747daf9e2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:52.293366 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15df83f-a508-4a4e-9343-c23bc5bece9d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:52.314916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 208/413] 10.222.0.91 () {58 vars in 1442 bytes} [Thu Sep 8 13:37:51 2022] PUT /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0/groups/8593666e58f84301a8aa5e475a82089e/roles/1a62bc555eb345fc8679cfa677069fd7 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:52.314916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b65eb-f55c-4e9e-8594-f4eb03f305f5 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:52.314916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b65eb-f55c-4e9e-8594-f4eb03f305f5 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:52.314916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b65eb-f55c-4e9e-8594-f4eb03f305f5 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:52.314916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 209/414] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:52.335608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052ec3c-d87a-4b80-a78e-c06475f637b4 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:52.335922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052ec3c-d87a-4b80-a78e-c06475f637b4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:52.336165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052ec3c-d87a-4b80-a78e-c06475f637b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:52.365683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c052ec3c-d87a-4b80-a78e-c06475f637b4 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:52.369020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c052ec3c-d87a-4b80-a78e-c06475f637b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:52.369791 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ce211-0528-4cac-820f-6045865ccee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:52.370466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ce211-0528-4cac-820f-6045865ccee6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:52.370786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ce211-0528-4cac-820f-6045865ccee6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:52.371144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ce211-0528-4cac-820f-6045865ccee6 None admin] PATH_INFO: `/v3/projects/189571d729d54bff900838adb0030c5b` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:52.372407 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ce211-0528-4cac-820f-6045865ccee6 None admin] RBAC: Authorizing `identity:get_project(project_id=189571d729d54bff900838adb0030c5b)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:52.373977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ce211-0528-4cac-820f-6045865ccee6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:52.376685 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 206/415] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:52 2022] GET /identity/v3/projects/189571d729d54bff900838adb0030c5b => generated 344 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:52.382761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ce211-0528-4cac-820f-6045865ccee6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:52.489725 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:52.541500 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ce211-0528-4cac-820f-6045865ccee6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:52.545819 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 210/416] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:52.594195 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de187e-d6bf-40b4-8cf2-4afab34c6f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:52.595435 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de187e-d6bf-40b4-8cf2-4afab34c6f9f None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:52.597346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de187e-d6bf-40b4-8cf2-4afab34c6f9f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:52.597346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de187e-d6bf-40b4-8cf2-4afab34c6f9f None admin] PATH_INFO: `/v3/projects/189571d729d54bff900838adb0030c5b/groups/1b4c2c5b07034085a5cb58d9bbf243f6/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:52.598440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de187e-d6bf-40b4-8cf2-4afab34c6f9f None admin] RBAC: Authorizing `identity:create_grant(project_id=189571d729d54bff900838adb0030c5b, group_id=1b4c2c5b07034085a5cb58d9bbf243f6, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:52.600202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de187e-d6bf-40b4-8cf2-4afab34c6f9f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:52.738816 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 207/417] 10.222.0.91 () {58 vars in 1442 bytes} [Thu Sep 8 13:37:52 2022] PUT /identity/v3/projects/189571d729d54bff900838adb0030c5b/groups/1b4c2c5b07034085a5cb58d9bbf243f6/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:52.939864 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9e2b9-295e-4ef7-bd0a-37188d556689 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:52.940374 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9e2b9-295e-4ef7-bd0a-37188d556689 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:52.940712 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9e2b9-295e-4ef7-bd0a-37188d556689 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:52.943489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 211/418] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:52.966082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf01f7-9c22-4836-b04c-2c98c75ad4c7 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:52.966578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf01f7-9c22-4836-b04c-2c98c75ad4c7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:52.967037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf01f7-9c22-4836-b04c-2c98c75ad4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:53.014406 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-12bf01f7-9c22-4836-b04c-2c98c75ad4c7 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:53.017777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-12bf01f7-9c22-4836-b04c-2c98c75ad4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:53.109257 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 208/419] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:53.157531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9e2b9-295e-4ef7-bd0a-37188d556689 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:53.226050 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:53.281852 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9e2b9-295e-4ef7-bd0a-37188d556689 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:53.333064 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed7ea3-aa93-4138-875f-981d357f5b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:53.333688 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed7ea3-aa93-4138-875f-981d357f5b8c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:53.333961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed7ea3-aa93-4138-875f-981d357f5b8c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:53.334224 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed7ea3-aa93-4138-875f-981d357f5b8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:53.345953 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed7ea3-aa93-4138-875f-981d357f5b8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:53.350760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-f7ed7ea3-aa93-4138-875f-981d357f5b8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:53.350760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:53.350760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:53.350760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:53.350760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:53.350760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:53.350760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:53.350760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:53.350760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:53.350760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:53.350760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:53.350760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:53.350760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:53.350760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:53.350760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:53.350760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:53.357406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:53.357406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:53.357406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:53.357406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:53.357406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:53.357406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:53.357406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:53.357406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:53.357406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:53.357406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:53.357406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:53.357406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:53.357406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:53.357406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:53.357406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:53.357406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:53.358206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:53.358206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:53.358206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:53.358206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:53.358206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:53.358206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:53.358206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:53.358206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:53.358206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:53.358206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:53.358206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:53.358206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:53.358206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 212/420] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:53.360241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf01f7-9c22-4836-b04c-2c98c75ad4c7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:53.397438 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed7ea3-aa93-4138-875f-981d357f5b8c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:53.452038 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:53.561663 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed7ea3-aa93-4138-875f-981d357f5b8c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:53.566749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf01f7-9c22-4836-b04c-2c98c75ad4c7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:53.597000 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:53.614368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ed90f-4295-403d-af39-71b07cc70049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:53.615428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ed90f-4295-403d-af39-71b07cc70049 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:53.615866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ed90f-4295-403d-af39-71b07cc70049 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:53.616297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ed90f-4295-403d-af39-71b07cc70049 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:53.629780 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ed90f-4295-403d-af39-71b07cc70049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:53.637090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-2d6ed90f-4295-403d-af39-71b07cc70049 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:53.637090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:53.637090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:53.637090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:53.637090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:53.637090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:53.637090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:53.637090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:53.637090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:53.637090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:53.637090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:53.637090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:53.637090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:53.637090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:53.637090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:53.637090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:53.637970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:53.637970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:53.637970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:53.637970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:53.637970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:53.637970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:53.637970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:53.637970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:53.637970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:53.637970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:53.637970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:53.637970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:53.637970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:53.637970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:53.637970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:53.637970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:53.638745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:53.638745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:53.638745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:53.638745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:53.638745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:53.638745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:53.638745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:53.638745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:53.638745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:53.638745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:53.638745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:53.638745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:53.646594 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 213/421] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:37:53.646594 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46fd34-3c35-4b37-8675-8f7727afb7fd None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:53.646594 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46fd34-3c35-4b37-8675-8f7727afb7fd None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:53.646594 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46fd34-3c35-4b37-8675-8f7727afb7fd None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:53.648147 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 214/422] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:53.648841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46fd34-3c35-4b37-8675-8f7727afb7fd None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:53.650630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3dc9c-f935-41a0-997e-e633491b0b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:53.655241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3dc9c-f935-41a0-997e-e633491b0b5a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:53.655526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3dc9c-f935-41a0-997e-e633491b0b5a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:53.655806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3dc9c-f935-41a0-997e-e633491b0b5a None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:53.657025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3dc9c-f935-41a0-997e-e633491b0b5a None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:53.657328 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3dc9c-f935-41a0-997e-e633491b0b5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:53.658716 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3dc9c-f935-41a0-997e-e633491b0b5a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:53.672981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 209/423] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:53.672981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8ede1-8536-4293-80e1-a39522c7d21e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:53.672981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8ede1-8536-4293-80e1-a39522c7d21e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:53.672981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8ede1-8536-4293-80e1-a39522c7d21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:53.690230 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a9b8ede1-8536-4293-80e1-a39522c7d21e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:53.691471 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a9b8ede1-8536-4293-80e1-a39522c7d21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:53.774978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 210/424] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:53.776352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8ede1-8536-4293-80e1-a39522c7d21e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:53.784816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46fd34-3c35-4b37-8675-8f7727afb7fd None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:53.825238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3915a-0926-48a4-97b4-e7816bf47422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:53.826196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3915a-0926-48a4-97b4-e7816bf47422 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:53.826683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3915a-0926-48a4-97b4-e7816bf47422 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:53.827167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3915a-0926-48a4-97b4-e7816bf47422 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:53.828842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3915a-0926-48a4-97b4-e7816bf47422 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:53.829393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3915a-0926-48a4-97b4-e7816bf47422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:53.831074 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3915a-0926-48a4-97b4-e7816bf47422 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:53.841833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 215/425] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:53.848933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3915a-0926-48a4-97b4-e7816bf47422 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:53.899199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:53.902873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:53.965651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3915a-0926-48a4-97b4-e7816bf47422 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:54.003156 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982f24e-cb7e-470a-ae0d-b457b865196c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:54.004228 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982f24e-cb7e-470a-ae0d-b457b865196c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:54.004731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982f24e-cb7e-470a-ae0d-b457b865196c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:54.005195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982f24e-cb7e-470a-ae0d-b457b865196c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:54.021857 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8ede1-8536-4293-80e1-a39522c7d21e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:54.077306 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982f24e-cb7e-470a-ae0d-b457b865196c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:54.080083 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982f24e-cb7e-470a-ae0d-b457b865196c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:54.083955 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424b657-38a7-4b6c-9c5c-d9bd006a39f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:54.084207 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424b657-38a7-4b6c-9c5c-d9bd006a39f0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:54.084436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424b657-38a7-4b6c-9c5c-d9bd006a39f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:54.084708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1424b657-38a7-4b6c-9c5c-d9bd006a39f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:54.123303 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:54.167944 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1424b657-38a7-4b6c-9c5c-d9bd006a39f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:54.169418 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1424b657-38a7-4b6c-9c5c-d9bd006a39f0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:54.199251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 216/426] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:37:54.200624 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982f24e-cb7e-470a-ae0d-b457b865196c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:54.217183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:54.281104 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 211/427] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:54.281104 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424b657-38a7-4b6c-9c5c-d9bd006a39f0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:54.319278 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982f24e-cb7e-470a-ae0d-b457b865196c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:54.360238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0339437-0dce-4012-81cf-3bdc589c79fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:54.361283 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0339437-0dce-4012-81cf-3bdc589c79fc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:54.361773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0339437-0dce-4012-81cf-3bdc589c79fc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:54.362250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0339437-0dce-4012-81cf-3bdc589c79fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:54.370016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:54.374591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0339437-0dce-4012-81cf-3bdc589c79fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:54.379969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-c0339437-0dce-4012-81cf-3bdc589c79fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:54.379969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:54.379969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:54.379969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:54.379969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:54.379969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:54.379969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:54.379969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:54.379969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:54.379969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:54.379969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:54.379969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:54.379969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:54.379969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:54.379969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:54.379969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:54.388779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:54.388779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:54.388779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:54.388779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:54.388779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:54.388779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:54.388779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:54.388779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:54.388779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:54.388779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:54.388779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:54.388779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:54.388779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:54.388779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:54.388779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:54.388779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:54.389560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:54.389560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:54.389560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:54.389560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:54.389560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:54.389560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:54.389560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:54.389560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:54.389560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:54.389560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:54.389560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:54.389560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:54.389560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 217/428] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:54.389560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0339437-0dce-4012-81cf-3bdc589c79fc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:54.390255 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1424b657-38a7-4b6c-9c5c-d9bd006a39f0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:54.427137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a6872-0cdf-4719-b620-4e518cd089f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:54.427865 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a6872-0cdf-4719-b620-4e518cd089f6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:54.428274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a6872-0cdf-4719-b620-4e518cd089f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:54.428671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a6872-0cdf-4719-b620-4e518cd089f6 None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:54.429870 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a6872-0cdf-4719-b620-4e518cd089f6 None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:54.431477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a6872-0cdf-4719-b620-4e518cd089f6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:54.437924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 212/429] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:54 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:54.437924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a6872-0cdf-4719-b620-4e518cd089f6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:54.454829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:54.510322 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0339437-0dce-4012-81cf-3bdc589c79fc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:54.552033 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f965e72-37b4-4b88-862f-0932cbc7868b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:54.552972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f965e72-37b4-4b88-862f-0932cbc7868b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:54.553350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f965e72-37b4-4b88-862f-0932cbc7868b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:54.553643 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f965e72-37b4-4b88-862f-0932cbc7868b None admin] PATH_INFO: `/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:54.555101 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f965e72-37b4-4b88-862f-0932cbc7868b None admin] RBAC: Authorizing `identity:get_project(project_id=7fc04ff6a9f94d04af0a564c8a1bab72)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:54.555668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a6872-0cdf-4719-b620-4e518cd089f6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:54.556605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f965e72-37b4-4b88-862f-0932cbc7868b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:54.564137 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 218/430] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:54 2022] GET /identity/v3/projects/7fc04ff6a9f94d04af0a564c8a1bab72 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:54.565189 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f965e72-37b4-4b88-862f-0932cbc7868b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:54.595571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698dffc-fc19-42fe-8716-bfd389bc0024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:54.596560 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698dffc-fc19-42fe-8716-bfd389bc0024 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:54.597098 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698dffc-fc19-42fe-8716-bfd389bc0024 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:54.597584 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698dffc-fc19-42fe-8716-bfd389bc0024 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:54.598446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698dffc-fc19-42fe-8716-bfd389bc0024 None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:54.599004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698dffc-fc19-42fe-8716-bfd389bc0024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:54.600570 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698dffc-fc19-42fe-8716-bfd389bc0024 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:54.610445 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 213/431] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:54.611576 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698dffc-fc19-42fe-8716-bfd389bc0024 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:54.651467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:54.694038 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f965e72-37b4-4b88-862f-0932cbc7868b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:54.696253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:54.722102 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698dffc-fc19-42fe-8716-bfd389bc0024 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:54.738467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35c47b-2f16-48a8-8b99-fcceaff386b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:54.739792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35c47b-2f16-48a8-8b99-fcceaff386b0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:54.740264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35c47b-2f16-48a8-8b99-fcceaff386b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:54.740685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35c47b-2f16-48a8-8b99-fcceaff386b0 None admin] PATH_INFO: `/v3/groups/8593666e58f84301a8aa5e475a82089e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:54.742306 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35c47b-2f16-48a8-8b99-fcceaff386b0 None admin] RBAC: Authorizing `identity:get_group(group_id=8593666e58f84301a8aa5e475a82089e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:54.744520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35c47b-2f16-48a8-8b99-fcceaff386b0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:54.753816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 219/432] 10.222.0.91 () {56 vars in 1180 bytes} [Thu Sep 8 13:37:54 2022] GET /identity/v3/groups/8593666e58f84301a8aa5e475a82089e => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:54.755459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35c47b-2f16-48a8-8b99-fcceaff386b0 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:54.764862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a4335-3a67-4ed3-be41-8a923738d3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:54.765601 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a4335-3a67-4ed3-be41-8a923738d3d9 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:54.765989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a4335-3a67-4ed3-be41-8a923738d3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:54.766315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a4335-3a67-4ed3-be41-8a923738d3d9 None admin] PATH_INFO: `/v3/groups/8593666e58f84301a8aa5e475a82089e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:54.767492 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a4335-3a67-4ed3-be41-8a923738d3d9 None admin] RBAC: Authorizing `identity:get_group(group_id=8593666e58f84301a8aa5e475a82089e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:54.769121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a4335-3a67-4ed3-be41-8a923738d3d9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:54.776769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 214/433] 10.222.0.91 () {56 vars in 1180 bytes} [Thu Sep 8 13:37:54 2022] GET /identity/v3/groups/8593666e58f84301a8aa5e475a82089e => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:54.776769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a4335-3a67-4ed3-be41-8a923738d3d9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:54.883794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35c47b-2f16-48a8-8b99-fcceaff386b0 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:54.884650 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a4335-3a67-4ed3-be41-8a923738d3d9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:54.919725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb3c4d-5580-481d-9f67-053ef5dcac32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:54.920323 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb3c4d-5580-481d-9f67-053ef5dcac32 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:54.920600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb3c4d-5580-481d-9f67-053ef5dcac32 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:54.920895 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb3c4d-5580-481d-9f67-053ef5dcac32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:54.923642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:54.923967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc87f7d-9998-477a-85ce-589e2328067b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:54.925029 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc87f7d-9998-477a-85ce-589e2328067b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:54.925029 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc87f7d-9998-477a-85ce-589e2328067b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:54.925277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc87f7d-9998-477a-85ce-589e2328067b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:54.926138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc87f7d-9998-477a-85ce-589e2328067b None admin] RBAC: Adding query filter params (group.id=8593666e58f84301a8aa5e475a82089e, role.id=1a62bc555eb345fc8679cfa677069fd7, scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:54.926449 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc87f7d-9998-477a-85ce-589e2328067b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:54.927970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc87f7d-9998-477a-85ce-589e2328067b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:54.934492 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 215/434] 10.222.0.91 () {56 vars in 1376 bytes} [Thu Sep 8 13:37:54 2022] GET /identity/v3/role_assignments?group.id=8593666e58f84301a8aa5e475a82089e&scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72&role.id=1a62bc555eb345fc8679cfa677069fd7 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:54.935249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc87f7d-9998-477a-85ce-589e2328067b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:54.949462 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:55.028453 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb3c4d-5580-481d-9f67-053ef5dcac32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:55.030669 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb3c4d-5580-481d-9f67-053ef5dcac32 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:55.064949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc87f7d-9998-477a-85ce-589e2328067b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:55.117003 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500229b-87e7-4699-a452-4ebc1541547d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:55.118253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500229b-87e7-4699-a452-4ebc1541547d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:55.118742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500229b-87e7-4699-a452-4ebc1541547d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:55.119957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500229b-87e7-4699-a452-4ebc1541547d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:55.120702 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500229b-87e7-4699-a452-4ebc1541547d None admin] RBAC: Adding query filter params (group.id=8593666e58f84301a8aa5e475a82089e, role.id=cb722313950945e5b147e994ec678cd1, scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:55.121026 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500229b-87e7-4699-a452-4ebc1541547d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:55.123660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7500229b-87e7-4699-a452-4ebc1541547d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:55.132739 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 216/435] 10.222.0.91 () {56 vars in 1376 bytes} [Thu Sep 8 13:37:54 2022] GET /identity/v3/role_assignments?group.id=8593666e58f84301a8aa5e475a82089e&scope.project.id=7fc04ff6a9f94d04af0a564c8a1bab72&role.id=cb722313950945e5b147e994ec678cd1 => generated 611 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:55.134579 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36872681-4c98-43ec-bf84-27b1118d0665 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:55.134713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36872681-4c98-43ec-bf84-27b1118d0665 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:55.134964 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36872681-4c98-43ec-bf84-27b1118d0665 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:55.137204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 217/436] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:55.144571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5627e-0ca6-4639-8bd1-9e220d2a1dba None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:55.144910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5627e-0ca6-4639-8bd1-9e220d2a1dba None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:55.145128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5627e-0ca6-4639-8bd1-9e220d2a1dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:55.170323 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b6e5627e-0ca6-4639-8bd1-9e220d2a1dba None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:55.172395 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b6e5627e-0ca6-4639-8bd1-9e220d2a1dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:55.175079 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 220/437] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:55.180816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb3c4d-5580-481d-9f67-053ef5dcac32 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:55.240514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:55.281617 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 218/438] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:55.330765 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb3c4d-5580-481d-9f67-053ef5dcac32 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:55.371274 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35150849-a447-4c19-a872-a65b9f4c670b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:55.371902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35150849-a447-4c19-a872-a65b9f4c670b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:55.372174 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35150849-a447-4c19-a872-a65b9f4c670b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:55.372440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35150849-a447-4c19-a872-a65b9f4c670b None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:55.373553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35150849-a447-4c19-a872-a65b9f4c670b None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:55.375032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35150849-a447-4c19-a872-a65b9f4c670b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:55.376983 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 221/439] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:55 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:55.382451 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5627e-0ca6-4639-8bd1-9e220d2a1dba None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:55.385547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:55.502025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5627e-0ca6-4639-8bd1-9e220d2a1dba None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:55.527652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788a980-7d2c-4ba3-bd1e-3e26008de631 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:55.527652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788a980-7d2c-4ba3-bd1e-3e26008de631 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:55.527916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788a980-7d2c-4ba3-bd1e-3e26008de631 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:55.530151 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 222/440] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:55.535040 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbf1cd-df95-483c-8144-685866233d43 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:55.535436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbf1cd-df95-483c-8144-685866233d43 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:55.535724 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbf1cd-df95-483c-8144-685866233d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:55.538944 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45096818-a179-46c6-a2d7-7dcc590df816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:55.539586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45096818-a179-46c6-a2d7-7dcc590df816 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:55.539861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45096818-a179-46c6-a2d7-7dcc590df816 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:55.540126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45096818-a179-46c6-a2d7-7dcc590df816 None admin] PATH_INFO: `/v3/groups/8593666e58f84301a8aa5e475a82089e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:55.541233 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45096818-a179-46c6-a2d7-7dcc590df816 None admin] RBAC: Authorizing `identity:get_group(group_id=8593666e58f84301a8aa5e475a82089e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:55.542789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45096818-a179-46c6-a2d7-7dcc590df816 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:55.544676 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 219/441] 10.222.0.91 () {56 vars in 1180 bytes} [Thu Sep 8 13:37:55 2022] GET /identity/v3/groups/8593666e58f84301a8aa5e475a82089e => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:55.550853 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45096818-a179-46c6-a2d7-7dcc590df816 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:55.557990 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c9dbf1cd-df95-483c-8144-685866233d43 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:55.559290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c9dbf1cd-df95-483c-8144-685866233d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:55.630095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:55.645818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 223/442] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:55.645818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbf1cd-df95-483c-8144-685866233d43 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:55.668157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45096818-a179-46c6-a2d7-7dcc590df816 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:55.686354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:55.714032 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702013d-db9d-4e93-9aa9-15c6a498a92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:55.714679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702013d-db9d-4e93-9aa9-15c6a498a92c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:55.717640 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702013d-db9d-4e93-9aa9-15c6a498a92c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:55.717640 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702013d-db9d-4e93-9aa9-15c6a498a92c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:55.717640 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702013d-db9d-4e93-9aa9-15c6a498a92c None admin] RBAC: Adding query filter params (group.id=8593666e58f84301a8aa5e475a82089e, role.id=cb722313950945e5b147e994ec678cd1, scope.project.id=8c60d15f044842a0bc451a2e001075f0) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:55.717640 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702013d-db9d-4e93-9aa9-15c6a498a92c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:55.718396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702013d-db9d-4e93-9aa9-15c6a498a92c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:55.728011 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 220/443] 10.222.0.91 () {56 vars in 1376 bytes} [Thu Sep 8 13:37:55 2022] GET /identity/v3/role_assignments?group.id=8593666e58f84301a8aa5e475a82089e&scope.project.id=8c60d15f044842a0bc451a2e001075f0&role.id=cb722313950945e5b147e994ec678cd1 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:55.778842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbf1cd-df95-483c-8144-685866233d43 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:55.824377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1429d-6de8-41bd-a881-64d73a850a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:55.825329 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1429d-6de8-41bd-a881-64d73a850a95 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:55.825911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1429d-6de8-41bd-a881-64d73a850a95 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:55.826371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af1429d-6de8-41bd-a881-64d73a850a95 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:55.840259 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af1429d-6de8-41bd-a881-64d73a850a95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:55.846297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-4af1429d-6de8-41bd-a881-64d73a850a95 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:55.846297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:55.846297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:55.846297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:55.846297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:55.846297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:55.846297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:55.846297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:55.846297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:55.846297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:55.846297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:55.846297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:55.846297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:55.846297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:55.846297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:55.846297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:55.847506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:55.847506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:55.847506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:55.847506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:55.847506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:55.847506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:55.847506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:55.847506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:55.847506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:55.847506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:55.847506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:55.847506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:55.847506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:55.847506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:55.847506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:55.847506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:55.848501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:55.848501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:55.848501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:55.848501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:55.848501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:55.848501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:55.848501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:55.848501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:55.848501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:55.848501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:55.848501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:55.848501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:55.850673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 224/444] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:55.857083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702013d-db9d-4e93-9aa9-15c6a498a92c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:55.933792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1429d-6de8-41bd-a881-64d73a850a95 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:55.992599 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702013d-db9d-4e93-9aa9-15c6a498a92c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:56.041732 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548db6d0-4bc3-4c52-a408-a5e8f459b471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:56.042590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548db6d0-4bc3-4c52-a408-a5e8f459b471 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:56.043115 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548db6d0-4bc3-4c52-a408-a5e8f459b471 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:56.043592 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548db6d0-4bc3-4c52-a408-a5e8f459b471 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:56.044722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548db6d0-4bc3-4c52-a408-a5e8f459b471 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:56.045099 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548db6d0-4bc3-4c52-a408-a5e8f459b471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:56.046550 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548db6d0-4bc3-4c52-a408-a5e8f459b471 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:56.048262 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:56.058790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:56.066926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 221/445] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:56.069184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af1429d-6de8-41bd-a881-64d73a850a95 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:56.071670 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548db6d0-4bc3-4c52-a408-a5e8f459b471 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:56.116933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6f074-17e0-44eb-b9e3-4f8d54a2c246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:56.117596 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6f074-17e0-44eb-b9e3-4f8d54a2c246 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:56.117866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6f074-17e0-44eb-b9e3-4f8d54a2c246 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:56.118137 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6f074-17e0-44eb-b9e3-4f8d54a2c246 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:56.134426 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6f074-17e0-44eb-b9e3-4f8d54a2c246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:56.139373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-e6c6f074-17e0-44eb-b9e3-4f8d54a2c246 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:56.139373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:56.139373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:56.139373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:56.139373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:56.139373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:56.139373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:56.139373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:56.139373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:56.139373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:56.139373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:56.139373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:56.139373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:56.139373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:56.139373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:56.139373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:56.140266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:56.140266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:56.140266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:56.140266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:56.140266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:56.140266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:56.140266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:56.140266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:56.140266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:56.140266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:56.140266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:56.140266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:56.140266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:56.140266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:56.140266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:56.140266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:56.141242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:56.141242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:56.141242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:56.141242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:56.141242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:56.141242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:56.141242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:56.141242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:56.141242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:56.141242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:56.141242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:56.141242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:37:56.141242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 225/446] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:37:56.147931 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6f074-17e0-44eb-b9e3-4f8d54a2c246 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:56.213373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548db6d0-4bc3-4c52-a408-a5e8f459b471 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:56.248265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bc451-c676-4abf-84eb-d95244cc3dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:56.248682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bc451-c676-4abf-84eb-d95244cc3dde None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:56.248927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bc451-c676-4abf-84eb-d95244cc3dde None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:56.249192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bc451-c676-4abf-84eb-d95244cc3dde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:56.286135 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6f074-17e0-44eb-b9e3-4f8d54a2c246 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:56.321092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:56.321673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4abde0-9e2c-4d85-b5fb-101866bebbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:56.322298 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4abde0-9e2c-4d85-b5fb-101866bebbc5 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:56.322570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4abde0-9e2c-4d85-b5fb-101866bebbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:56.322836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4abde0-9e2c-4d85-b5fb-101866bebbc5 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:56.323543 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4abde0-9e2c-4d85-b5fb-101866bebbc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:56.323842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4abde0-9e2c-4d85-b5fb-101866bebbc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:56.325245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4abde0-9e2c-4d85-b5fb-101866bebbc5 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:56.329403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bc451-c676-4abf-84eb-d95244cc3dde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:56.330860 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bc451-c676-4abf-84eb-d95244cc3dde None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:56.334960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 226/447] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:56.341216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4abde0-9e2c-4d85-b5fb-101866bebbc5 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:56.364645 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:56.450022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 222/448] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:37:56.452669 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4abde0-9e2c-4d85-b5fb-101866bebbc5 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:56.456209 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bc451-c676-4abf-84eb-d95244cc3dde None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:56.497961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21ffc3-d6ce-4785-8886-f12bced29ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:56.498568 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21ffc3-d6ce-4785-8886-f12bced29ff4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:56.498849 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21ffc3-d6ce-4785-8886-f12bced29ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:56.499173 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21ffc3-d6ce-4785-8886-f12bced29ff4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:56.554053 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:56.560207 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bc451-c676-4abf-84eb-d95244cc3dde None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:56.570527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21ffc3-d6ce-4785-8886-f12bced29ff4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:56.572019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21ffc3-d6ce-4785-8886-f12bced29ff4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:56.588132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:56.599635 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfa3d9-fdfb-403b-8dd8-91ddbbc99456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:56.600669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfa3d9-fdfb-403b-8dd8-91ddbbc99456 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:56.600853 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfa3d9-fdfb-403b-8dd8-91ddbbc99456 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:56.601123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfa3d9-fdfb-403b-8dd8-91ddbbc99456 None admin] PATH_INFO: `/v3/projects/8c60d15f044842a0bc451a2e001075f0` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:56.602445 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfa3d9-fdfb-403b-8dd8-91ddbbc99456 None admin] RBAC: Authorizing `identity:get_project(project_id=8c60d15f044842a0bc451a2e001075f0)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:56.603775 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfa3d9-fdfb-403b-8dd8-91ddbbc99456 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:56.605494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 223/449] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:56 2022] GET /identity/v3/projects/8c60d15f044842a0bc451a2e001075f0 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:56.611450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfa3d9-fdfb-403b-8dd8-91ddbbc99456 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:56.687375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 227/450] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:37:56.693269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21ffc3-d6ce-4785-8886-f12bced29ff4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:56.715844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfa3d9-fdfb-403b-8dd8-91ddbbc99456 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:56.751716 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52423bc4-6ca2-4466-a7b6-30ed7a34267c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:56.752325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52423bc4-6ca2-4466-a7b6-30ed7a34267c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:56.752609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52423bc4-6ca2-4466-a7b6-30ed7a34267c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:56.752881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52423bc4-6ca2-4466-a7b6-30ed7a34267c None admin] PATH_INFO: `/v3/groups/8593666e58f84301a8aa5e475a82089e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:56.754056 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52423bc4-6ca2-4466-a7b6-30ed7a34267c None admin] RBAC: Authorizing `identity:get_group(group_id=8593666e58f84301a8aa5e475a82089e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:56.755524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52423bc4-6ca2-4466-a7b6-30ed7a34267c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:56.757091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 224/451] 10.222.0.91 () {56 vars in 1180 bytes} [Thu Sep 8 13:37:56 2022] GET /identity/v3/groups/8593666e58f84301a8aa5e475a82089e => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:56.762984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52423bc4-6ca2-4466-a7b6-30ed7a34267c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:56.793885 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21ffc3-d6ce-4785-8886-f12bced29ff4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:56.795495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:56.810856 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:56.835431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe4b38-94ad-475d-a21b-a5c0c012991f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:56.836071 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe4b38-94ad-475d-a21b-a5c0c012991f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:56.836323 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe4b38-94ad-475d-a21b-a5c0c012991f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:56.836596 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe4b38-94ad-475d-a21b-a5c0c012991f None admin] PATH_INFO: `/v3/projects/189571d729d54bff900838adb0030c5b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:56.837745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe4b38-94ad-475d-a21b-a5c0c012991f None admin] RBAC: Authorizing `identity:get_project(project_id=189571d729d54bff900838adb0030c5b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:56.839225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe4b38-94ad-475d-a21b-a5c0c012991f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:56.840856 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 228/452] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:56 2022] GET /identity/v3/projects/189571d729d54bff900838adb0030c5b => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:56.847482 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe4b38-94ad-475d-a21b-a5c0c012991f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:56.876193 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52423bc4-6ca2-4466-a7b6-30ed7a34267c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:56.912885 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5982db-d28b-46ad-8a95-fd231de93e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:56.913494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5982db-d28b-46ad-8a95-fd231de93e7d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:56.913777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5982db-d28b-46ad-8a95-fd231de93e7d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:56.914051 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5982db-d28b-46ad-8a95-fd231de93e7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:56.914785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5982db-d28b-46ad-8a95-fd231de93e7d None admin] RBAC: Adding query filter params (group.id=8593666e58f84301a8aa5e475a82089e, role.id=1a62bc555eb345fc8679cfa677069fd7, scope.project.id=8c60d15f044842a0bc451a2e001075f0) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:56.915132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5982db-d28b-46ad-8a95-fd231de93e7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:56.916601 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5982db-d28b-46ad-8a95-fd231de93e7d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:56.923558 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 225/453] 10.222.0.91 () {56 vars in 1376 bytes} [Thu Sep 8 13:37:56 2022] GET /identity/v3/role_assignments?group.id=8593666e58f84301a8aa5e475a82089e&scope.project.id=8c60d15f044842a0bc451a2e001075f0&role.id=1a62bc555eb345fc8679cfa677069fd7 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:56.992211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe4b38-94ad-475d-a21b-a5c0c012991f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:57.049883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3635f-28c7-472a-8b8d-5adafae2bbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:57.050463 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3635f-28c7-472a-8b8d-5adafae2bbb3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:57.050740 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3635f-28c7-472a-8b8d-5adafae2bbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:57.051044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3635f-28c7-472a-8b8d-5adafae2bbb3 None admin] PATH_INFO: `/v3/groups/1b4c2c5b07034085a5cb58d9bbf243f6` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:57.052226 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3635f-28c7-472a-8b8d-5adafae2bbb3 None admin] RBAC: Authorizing `identity:get_group(group_id=1b4c2c5b07034085a5cb58d9bbf243f6)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:57.053822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3635f-28c7-472a-8b8d-5adafae2bbb3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:57.055561 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 229/454] 10.222.0.91 () {56 vars in 1180 bytes} [Thu Sep 8 13:37:56 2022] GET /identity/v3/groups/1b4c2c5b07034085a5cb58d9bbf243f6 => generated 229 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:57.061845 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5982db-d28b-46ad-8a95-fd231de93e7d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:57.197534 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:57.204801 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5982db-d28b-46ad-8a95-fd231de93e7d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:57.241344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc87ea-8cb5-479e-a67f-243920d59670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:57.241989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc87ea-8cb5-479e-a67f-243920d59670 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:57.242218 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc87ea-8cb5-479e-a67f-243920d59670 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:57.242490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc87ea-8cb5-479e-a67f-243920d59670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:57.243254 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc87ea-8cb5-479e-a67f-243920d59670 None admin] RBAC: Adding query filter params (group.id=1b4c2c5b07034085a5cb58d9bbf243f6, role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=189571d729d54bff900838adb0030c5b) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:57.243603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc87ea-8cb5-479e-a67f-243920d59670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:57.245151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc87ea-8cb5-479e-a67f-243920d59670 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:57.251268 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 226/455] 10.222.0.91 () {56 vars in 1376 bytes} [Thu Sep 8 13:37:57 2022] GET /identity/v3/role_assignments?group.id=1b4c2c5b07034085a5cb58d9bbf243f6&scope.project.id=189571d729d54bff900838adb0030c5b&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:00.207850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068669d-0af6-4d34-bbe5-a77ff03ba8ec None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:00.208801 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068669d-0af6-4d34-bbe5-a77ff03ba8ec None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:00.209279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068669d-0af6-4d34-bbe5-a77ff03ba8ec None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:00.211025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 230/456] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:00.217144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff0a8c-f1c2-486b-80a9-2bcbecb54d1b None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:00.217959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff0a8c-f1c2-486b-80a9-2bcbecb54d1b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:00.218398 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff0a8c-f1c2-486b-80a9-2bcbecb54d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:00.237560 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-78ff0a8c-f1c2-486b-80a9-2bcbecb54d1b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:00.238980 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-78ff0a8c-f1c2-486b-80a9-2bcbecb54d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:00.300124 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4137a9-96bf-4d93-a46d-47338aa2331f None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:00.300748 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4137a9-96bf-4d93-a46d-47338aa2331f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:00.301250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4137a9-96bf-4d93-a46d-47338aa2331f None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:00.302820 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 231/457] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:00.315004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4457a-ed09-4a66-8de5-8a4f1e786983 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:00.315646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4457a-ed09-4a66-8de5-8a4f1e786983 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:00.316171 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4457a-ed09-4a66-8de5-8a4f1e786983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:00.333635 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 227/458] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:00.347877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e6d3c-1e6f-4357-b1ac-933275bacc18 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:00.348443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e6d3c-1e6f-4357-b1ac-933275bacc18 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:00.348867 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e6d3c-1e6f-4357-b1ac-933275bacc18 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:00.350726 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 228/459] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:00.352931 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-00c4457a-ed09-4a66-8de5-8a4f1e786983 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:00.354273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-00c4457a-ed09-4a66-8de5-8a4f1e786983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:00.360760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd94c54-5241-4011-81d6-41d1dd8b5925 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:00.361309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd94c54-5241-4011-81d6-41d1dd8b5925 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:00.361767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd94c54-5241-4011-81d6-41d1dd8b5925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:00.389118 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-bfd94c54-5241-4011-81d6-41d1dd8b5925 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:00.390930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-bfd94c54-5241-4011-81d6-41d1dd8b5925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:00.451268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:00.498167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 232/460] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:00.516392 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 229/461] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:00.664298 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4457a-ed09-4a66-8de5-8a4f1e786983 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:00.808734 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4457a-ed09-4a66-8de5-8a4f1e786983 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:00.861065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf3e91-f673-45a6-b28a-2482fd12948d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:00.861938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf3e91-f673-45a6-b28a-2482fd12948d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:00.862356 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf3e91-f673-45a6-b28a-2482fd12948d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:00.862794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf3e91-f673-45a6-b28a-2482fd12948d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:00.881054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf3e91-f673-45a6-b28a-2482fd12948d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:00.882752 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf3e91-f673-45a6-b28a-2482fd12948d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:00.902878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-a8bf3e91-f673-45a6-b28a-2482fd12948d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:00.902878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:00.902878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:00.902878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:00.902878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:00.902878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:00.902878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:00.902878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:00.902878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:00.902878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:00.902878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:00.902878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:00.902878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:00.902878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:00.902878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:00.902878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:00.904025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:00.905047 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:00.905047 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:00.905047 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:00.906480 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 233/462] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:00.914322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd94c54-5241-4011-81d6-41d1dd8b5925 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:00.915447 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf3e91-f673-45a6-b28a-2482fd12948d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:00.951940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:00.955326 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:01.059862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf3e91-f673-45a6-b28a-2482fd12948d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:01.090122 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd94c54-5241-4011-81d6-41d1dd8b5925 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:01.104364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1129f4-48ee-4336-8299-a9191b43a203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:01.105032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1129f4-48ee-4336-8299-a9191b43a203 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.105312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1129f4-48ee-4336-8299-a9191b43a203 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.105585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1129f4-48ee-4336-8299-a9191b43a203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.119619 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1129f4-48ee-4336-8299-a9191b43a203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:01.121604 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1129f4-48ee-4336-8299-a9191b43a203 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:01.139763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-fe1129f4-48ee-4336-8299-a9191b43a203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:01.139763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:01.139763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:01.139763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:01.139763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:01.139763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:01.139763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:01.139763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:01.139763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:01.139763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:01.139763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:01.139763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:01.139763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:01.139763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:01.139763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:01.139763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:01.140728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:01.142124 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:01.142124 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:01.142124 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:01.142297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90953331-1f09-49a3-a501-825474268c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:01.142297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90953331-1f09-49a3-a501-825474268c1b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.142297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90953331-1f09-49a3-a501-825474268c1b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.142297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90953331-1f09-49a3-a501-825474268c1b None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.143072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90953331-1f09-49a3-a501-825474268c1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:01.143168 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90953331-1f09-49a3-a501-825474268c1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:01.144662 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 234/463] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:01.144662 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1129f4-48ee-4336-8299-a9191b43a203 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:01.144828 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90953331-1f09-49a3-a501-825474268c1b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:01.159842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 230/464] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:01.161300 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7582-cab0-4d52-8ff7-7aafa215798d None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.161574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7582-cab0-4d52-8ff7-7aafa215798d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.161941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7582-cab0-4d52-8ff7-7aafa215798d None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.163266 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 231/465] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:01.167119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed7582-cab0-4d52-8ff7-7aafa215798d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:01.245769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:01.260627 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:01.262363 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1129f4-48ee-4336-8299-a9191b43a203 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:01.284210 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed7582-cab0-4d52-8ff7-7aafa215798d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:01.297160 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe3e13-ff2a-4788-b14b-e005b2d90414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:01.297785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe3e13-ff2a-4788-b14b-e005b2d90414 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.298060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe3e13-ff2a-4788-b14b-e005b2d90414 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.298332 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe3e13-ff2a-4788-b14b-e005b2d90414 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.310168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe3e13-ff2a-4788-b14b-e005b2d90414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:01.311594 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe3e13-ff2a-4788-b14b-e005b2d90414 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:01.321460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e79d0-312e-45b4-8684-0e45320dd8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:01.322079 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e79d0-312e-45b4-8684-0e45320dd8a3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.322383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e79d0-312e-45b4-8684-0e45320dd8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.322659 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e79d0-312e-45b4-8684-0e45320dd8a3 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.323352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e79d0-312e-45b4-8684-0e45320dd8a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:01.323655 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e79d0-312e-45b4-8684-0e45320dd8a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:01.325174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e79d0-312e-45b4-8684-0e45320dd8a3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:01.326293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-6dfe3e13-ff2a-4788-b14b-e005b2d90414 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:01.326293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:01.326293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:01.326293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:01.326293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:01.326293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:01.326293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:01.326293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:01.326293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:01.326293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:01.326293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:01.326293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:01.326293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:01.326293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:01.326293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:01.326293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:01.327245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:01.328068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:01.328068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:01.328068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:01.328068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 235/466] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:01.328284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe3e13-ff2a-4788-b14b-e005b2d90414 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:01.341951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 232/467] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:01.343297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf88ad-dfa6-46b1-bc11-8ae0c005f969 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.343572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf88ad-dfa6-46b1-bc11-8ae0c005f969 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.343845 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf88ad-dfa6-46b1-bc11-8ae0c005f969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.365563 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-50cf88ad-dfa6-46b1-bc11-8ae0c005f969 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:01.366760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-50cf88ad-dfa6-46b1-bc11-8ae0c005f969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:01.435497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 233/468] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:01.436867 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc609f-c62e-47e3-95ce-44b98af96503 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.437157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc609f-c62e-47e3-95ce-44b98af96503 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.437450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc609f-c62e-47e3-95ce-44b98af96503 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.438710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 234/469] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:01.444102 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc609f-c62e-47e3-95ce-44b98af96503 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:01.445316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe3e13-ff2a-4788-b14b-e005b2d90414 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:01.492009 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b10ac8-741c-459a-bf34-ff8c3e462c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:01.492606 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b10ac8-741c-459a-bf34-ff8c3e462c3b None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.492887 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b10ac8-741c-459a-bf34-ff8c3e462c3b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.493158 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b10ac8-741c-459a-bf34-ff8c3e462c3b None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.493804 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b10ac8-741c-459a-bf34-ff8c3e462c3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:01.495269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b10ac8-741c-459a-bf34-ff8c3e462c3b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:01.505341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:01.532808 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:01.565123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc609f-c62e-47e3-95ce-44b98af96503 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:01.600104 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a4314e-bd3a-48cd-9a5e-c907e30dca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:01.600709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a4314e-bd3a-48cd-9a5e-c907e30dca00 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.601008 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a4314e-bd3a-48cd-9a5e-c907e30dca00 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.601279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a4314e-bd3a-48cd-9a5e-c907e30dca00 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.601925 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a4314e-bd3a-48cd-9a5e-c907e30dca00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:01.602226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a4314e-bd3a-48cd-9a5e-c907e30dca00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:01.603641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a4314e-bd3a-48cd-9a5e-c907e30dca00 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:01.614954 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 235/470] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:01.615758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a4314e-bd3a-48cd-9a5e-c907e30dca00 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:01.642669 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 236/471] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:01 2022] POST /identity/v3/users => generated 248 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:01.642669 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d529fc-75ed-474b-99f9-6d2c5c2b589b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.642669 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d529fc-75ed-474b-99f9-6d2c5c2b589b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.642669 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d529fc-75ed-474b-99f9-6d2c5c2b589b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.664970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e1d529fc-75ed-474b-99f9-6d2c5c2b589b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:01.667226 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e1d529fc-75ed-474b-99f9-6d2c5c2b589b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:01.764882 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a4314e-bd3a-48cd-9a5e-c907e30dca00 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:01.771068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 237/472] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:01.772113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d529fc-75ed-474b-99f9-6d2c5c2b589b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:01.809901 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:01.819864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401cf82-02ea-4eb0-8f64-db7072fe64e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:01.820388 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401cf82-02ea-4eb0-8f64-db7072fe64e3 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.820934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401cf82-02ea-4eb0-8f64-db7072fe64e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.821213 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401cf82-02ea-4eb0-8f64-db7072fe64e3 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.821898 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401cf82-02ea-4eb0-8f64-db7072fe64e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:01.824247 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401cf82-02ea-4eb0-8f64-db7072fe64e3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:01.902575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d529fc-75ed-474b-99f9-6d2c5c2b589b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:01.913228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 236/473] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:01 2022] POST /identity/v3/users => generated 246 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:01.914980 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401cf82-02ea-4eb0-8f64-db7072fe64e3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:01.938920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:01.946655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3118f4-8315-4ad7-81a0-9c3cd59b928d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:01.947343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3118f4-8315-4ad7-81a0-9c3cd59b928d None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.947607 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3118f4-8315-4ad7-81a0-9c3cd59b928d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.947948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3118f4-8315-4ad7-81a0-9c3cd59b928d None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.948630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3118f4-8315-4ad7-81a0-9c3cd59b928d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:01.950724 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3118f4-8315-4ad7-81a0-9c3cd59b928d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:02.066320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 238/474] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:01 2022] POST /identity/v3/users => generated 248 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:02.070260 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401cf82-02ea-4eb0-8f64-db7072fe64e3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:02.093868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3118f4-8315-4ad7-81a0-9c3cd59b928d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:02.149030 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023490cf-3b3c-42b7-bea6-83a7900fb3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:02.150605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023490cf-3b3c-42b7-bea6-83a7900fb3d8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:02.151298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023490cf-3b3c-42b7-bea6-83a7900fb3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:02.151788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023490cf-3b3c-42b7-bea6-83a7900fb3d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:02.167603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023490cf-3b3c-42b7-bea6-83a7900fb3d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:02.170789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023490cf-3b3c-42b7-bea6-83a7900fb3d8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:02.193382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-023490cf-3b3c-42b7-bea6-83a7900fb3d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:02.193382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:02.193382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:02.193382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:02.193382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:02.193382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:02.193382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:02.193382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:02.193382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:02.193382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:02.193382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:02.193382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:02.193382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:02.193382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:02.193382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:02.193382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:02.194345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:02.195200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:02.195200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:02.195200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:02.195200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 237/475] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:02.201312 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023490cf-3b3c-42b7-bea6-83a7900fb3d8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:02.255340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:02.278718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3118f4-8315-4ad7-81a0-9c3cd59b928d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:02.334440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023490cf-3b3c-42b7-bea6-83a7900fb3d8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:02.341197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e746600-fb13-4591-81a1-f041bbc99a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:02.341407 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e746600-fb13-4591-81a1-f041bbc99a24 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:02.341778 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e746600-fb13-4591-81a1-f041bbc99a24 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:02.342016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e746600-fb13-4591-81a1-f041bbc99a24 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:02.342706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e746600-fb13-4591-81a1-f041bbc99a24 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:02.345589 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e746600-fb13-4591-81a1-f041bbc99a24 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:02.379735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118f285-8e59-48e8-b29c-4af05f09996e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:02.380661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118f285-8e59-48e8-b29c-4af05f09996e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:02.381112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118f285-8e59-48e8-b29c-4af05f09996e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:02.381614 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118f285-8e59-48e8-b29c-4af05f09996e None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:02.382267 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118f285-8e59-48e8-b29c-4af05f09996e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:02.382570 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118f285-8e59-48e8-b29c-4af05f09996e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:02.384232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118f285-8e59-48e8-b29c-4af05f09996e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:02.398733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 238/476] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:02.401418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.sql.core [None req-7e746600-fb13-4591-81a1-f041bbc99a24 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 13:38:02.401418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 08 13:38:02.401418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [parameters: {'id': 'a3dbf6db92c54505b9cdba1cfd68203e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 08 13:38:02.401418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93173) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 13:38:02.407291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118f285-8e59-48e8-b29c-4af05f09996e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:02.422886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-7e746600-fb13-4591-81a1-f041bbc99a24 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 08 13:38:02.422886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:02.422886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:38:02.422886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:38:02.422886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:38:02.422886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:38:02.422886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:38:02.422886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:38:02.422886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:38:02.422886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:38:02.422886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:38:02.422886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:38:02.422886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:38:02.422886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result.read() Sep 08 13:38:02.422886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return ref.to_dict() Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:38:02.423833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application yield resource Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise self.value Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application session.commit() Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 13:38:02.424725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.session.flush() Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application compat.raise_( Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 13:38:02.425551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 13:38:02.426413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 13:38:02.426413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 13:38:02.426413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 08 13:38:02.426413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 08 13:38:02.426413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 13:38:02.426413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 13:38:02.426413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 13:38:02.426413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 13:38:02.426413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 13:38:02.426413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 13:38:02.426413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 13:38:02.426413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 13:38:02.426413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 13:38:02.426413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 13:38:02.426413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:38:02.426413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception Sep 08 13:38:02.429731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:38:02.429731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:38:02.429731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:38:02.429731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:38:02.429731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:38:02.429731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:38:02.429731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:38:02.429731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:38:02.429731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:38:02.429731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:38:02.429731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:38:02.429731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result.read() Sep 08 13:38:02.429731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:38:02.429731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:38:02.429731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:38:02.429731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:38:02.430535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:38:02.430535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:38:02.430535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:38:02.430535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:38:02.430535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 13:38:02.430535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 08 13:38:02.430535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [parameters: {'id': 'a3dbf6db92c54505b9cdba1cfd68203e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 08 13:38:02.430535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 13:38:02.430535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:02.430535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:02.430535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:02.430535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:02.430535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:02.430535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:02.430535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:02.430535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:02.431428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:02.431428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:02.431428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:02.431428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:02.431428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:02.431428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:02.431428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 08 13:38:02.431428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 08 13:38:02.431428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:02.431428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:02.431428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 08 13:38:02.431428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 08 13:38:02.431428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 13:38:02.431428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 13:38:02.431428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 08 13:38:02.431428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:02.435060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 239/477] 10.222.0.91 () {60 vars in 1131 bytes} [Thu Sep 8 13:38:02 2022] POST /identity/v3/roles => generated 144 bytes in 337 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:02.437334 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e746600-fb13-4591-81a1-f041bbc99a24 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:02.502884 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:02.559813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:02.565839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118f285-8e59-48e8-b29c-4af05f09996e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:02.569496 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e746600-fb13-4591-81a1-f041bbc99a24 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:02.611892 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d2c59-a184-42ca-8ac8-f12743559188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:02.612982 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d2c59-a184-42ca-8ac8-f12743559188 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:02.613390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d2c59-a184-42ca-8ac8-f12743559188 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:02.613750 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d2c59-a184-42ca-8ac8-f12743559188 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:02.617780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d821b-a851-411d-a4c0-95c163bc3ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:02.618687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d821b-a851-411d-a4c0-95c163bc3ce7 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:02.619188 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d821b-a851-411d-a4c0-95c163bc3ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:02.619602 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d821b-a851-411d-a4c0-95c163bc3ce7 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:02.620441 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d821b-a851-411d-a4c0-95c163bc3ce7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:02.622660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d821b-a851-411d-a4c0-95c163bc3ce7 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:02.629123 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d2c59-a184-42ca-8ac8-f12743559188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:02.635088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-a77d2c59-a184-42ca-8ac8-f12743559188 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:02.635088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:02.635088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:02.635088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:02.635088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:02.635088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:02.635088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:02.635088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:02.635088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:02.635088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:02.635088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:02.635088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:02.635088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:02.635088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:02.635088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:02.635088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:02.636063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:02.636063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:02.636063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:02.636063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:02.636063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:02.636063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:02.636063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:02.636063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:02.636063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:02.636063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:02.636063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:02.636063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:02.636063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:02.636063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:02.636063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:02.636063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:02.637039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:02.637039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:02.637039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:02.637039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:02.637039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:02.637039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:02.637039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:02.637039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:02.637039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:02.637039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:02.637039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:02.637039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:02.637039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 240/478] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:02.643541 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d2c59-a184-42ca-8ac8-f12743559188 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:02.699085 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 239/479] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:02 2022] POST /identity/v3/users => generated 249 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:38:02.788661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d2c59-a184-42ca-8ac8-f12743559188 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:02.790350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:02.828127 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb391e5-80c3-4ccf-af4b-e7352ffd235a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:02.828778 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb391e5-80c3-4ccf-af4b-e7352ffd235a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:02.830001 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb391e5-80c3-4ccf-af4b-e7352ffd235a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:02.830278 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb391e5-80c3-4ccf-af4b-e7352ffd235a None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:02.830965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb391e5-80c3-4ccf-af4b-e7352ffd235a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:02.831270 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb391e5-80c3-4ccf-af4b-e7352ffd235a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:02.832651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb391e5-80c3-4ccf-af4b-e7352ffd235a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:02.841710 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 241/480] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:04.565647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94a5ac-6040-4dc2-b8ed-ae2fe5f8c7ee None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:04.566657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94a5ac-6040-4dc2-b8ed-ae2fe5f8c7ee None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:04.567167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94a5ac-6040-4dc2-b8ed-ae2fe5f8c7ee None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:04.568766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 240/481] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:04.575655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906e4c4-d1ec-4593-afb2-358df11f64ce None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:04.576351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906e4c4-d1ec-4593-afb2-358df11f64ce None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:04.576432 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906e4c4-d1ec-4593-afb2-358df11f64ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:04.599974 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d906e4c4-d1ec-4593-afb2-358df11f64ce None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:04.601189 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d906e4c4-d1ec-4593-afb2-358df11f64ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:04.676668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 242/482] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:04.697584 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd84dca-12c7-4882-98fd-c86d0faa8a2e None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:04.697860 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd84dca-12c7-4882-98fd-c86d0faa8a2e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:04.698202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd84dca-12c7-4882-98fd-c86d0faa8a2e None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:04.699839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 241/483] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:04.706044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eefd81-2e66-49e0-b312-9186f688d79c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:04.706359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eefd81-2e66-49e0-b312-9186f688d79c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:04.706635 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eefd81-2e66-49e0-b312-9186f688d79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:04.726953 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-26eefd81-2e66-49e0-b312-9186f688d79c None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:04.728429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-26eefd81-2e66-49e0-b312-9186f688d79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:04.816610 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 243/484] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:04.989909 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd84dca-12c7-4882-98fd-c86d0faa8a2e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:05.131318 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eefd81-2e66-49e0-b312-9186f688d79c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:05.144144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd84dca-12c7-4882-98fd-c86d0faa8a2e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:05.175265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:05.210108 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7246e32-e697-4b19-881e-76f2d4aba690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:05.210787 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7246e32-e697-4b19-881e-76f2d4aba690 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:05.211205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7246e32-e697-4b19-881e-76f2d4aba690 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:05.211409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7246e32-e697-4b19-881e-76f2d4aba690 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:05.234208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7246e32-e697-4b19-881e-76f2d4aba690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:05.239637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-b7246e32-e697-4b19-881e-76f2d4aba690 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:05.239637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:05.239637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:05.239637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:05.239637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:05.239637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:05.239637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:05.239637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:05.239637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:05.239637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:05.239637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:05.239637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:05.239637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:05.239637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:05.239637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:05.239637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:05.240515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:05.240515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:05.240515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:05.240515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:05.240515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:05.240515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:05.240515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:05.240515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:05.240515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:05.240515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:05.240515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:05.240515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:05.240515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:05.240515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:05.240515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:05.240515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:05.241272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:05.241272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:05.241272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:05.241272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:05.241272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:05.241272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:05.241272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:05.241272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:05.241272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:05.241272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:05.241272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:05.241272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:05.242568 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 242/485] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:05.251727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7246e32-e697-4b19-881e-76f2d4aba690 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:05.289079 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:05.333295 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eefd81-2e66-49e0-b312-9186f688d79c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:05.408938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e2ce1-cf78-43c0-ae50-4486d9ac0f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:05.409527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e2ce1-cf78-43c0-ae50-4486d9ac0f0e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:05.410157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e2ce1-cf78-43c0-ae50-4486d9ac0f0e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:05.410443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e2ce1-cf78-43c0-ae50-4486d9ac0f0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:05.426844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e2ce1-cf78-43c0-ae50-4486d9ac0f0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:05.433231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7246e32-e697-4b19-881e-76f2d4aba690 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:05.433371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-679e2ce1-cf78-43c0-ae50-4486d9ac0f0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:05.433371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:05.433371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:05.433371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:05.433371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:05.433371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:05.433371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:05.433371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:05.433371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:05.433371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:05.433371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:05.433371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:05.433371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:05.433371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:05.433371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:05.433371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:05.434243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:05.434243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:05.434243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:05.434243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:05.434243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:05.434243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:05.434243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:05.434243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:05.434243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:05.434243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:05.434243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:05.434243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:05.434243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:05.434243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:05.434243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:05.434243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:05.435120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:05.435120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:05.435120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:05.435120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:05.435120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:05.435120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:05.435120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:05.435120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:05.435120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:05.435120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:05.435120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:05.435120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:05.439355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 244/486] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:05.441989 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4ddf2-8c76-4d20-9886-2b8771217f80 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:05.442297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4ddf2-8c76-4d20-9886-2b8771217f80 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:05.442588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4ddf2-8c76-4d20-9886-2b8771217f80 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:05.444162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 245/487] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:05.448252 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4ddf2-8c76-4d20-9886-2b8771217f80 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:05.488996 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de7da20-cc2b-4942-8d91-916e1056850f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:05.490017 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7da20-cc2b-4942-8d91-916e1056850f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:05.490579 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7da20-cc2b-4942-8d91-916e1056850f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:05.491159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de7da20-cc2b-4942-8d91-916e1056850f None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:05.492164 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7da20-cc2b-4942-8d91-916e1056850f None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:05.492726 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7da20-cc2b-4942-8d91-916e1056850f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:05.494722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de7da20-cc2b-4942-8d91-916e1056850f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:05.506110 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 243/488] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:05.511373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaee426-d1c9-4a7f-9b1f-02762e9651d6 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:05.511373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaee426-d1c9-4a7f-9b1f-02762e9651d6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:05.511373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaee426-d1c9-4a7f-9b1f-02762e9651d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:05.528611 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-aaaee426-d1c9-4a7f-9b1f-02762e9651d6 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:05.530181 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-aaaee426-d1c9-4a7f-9b1f-02762e9651d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:05.555924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:05.574759 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4ddf2-8c76-4d20-9886-2b8771217f80 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:05.615214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246ab17-e20e-4e94-8e34-22c9435ad43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:05.615920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246ab17-e20e-4e94-8e34-22c9435ad43c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:05.616131 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246ab17-e20e-4e94-8e34-22c9435ad43c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:05.616366 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246ab17-e20e-4e94-8e34-22c9435ad43c None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:05.616989 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246ab17-e20e-4e94-8e34-22c9435ad43c None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:05.617365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246ab17-e20e-4e94-8e34-22c9435ad43c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:05.618625 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 244/489] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:05.619102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246ab17-e20e-4e94-8e34-22c9435ad43c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:05.621291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaee426-d1c9-4a7f-9b1f-02762e9651d6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:05.628312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 246/490] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:05.629616 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b4bdf-57b4-4332-81b2-15881f6d1277 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:05.629896 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b4bdf-57b4-4332-81b2-15881f6d1277 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:05.630162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b4bdf-57b4-4332-81b2-15881f6d1277 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:05.631625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 247/491] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:05.633611 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b4bdf-57b4-4332-81b2-15881f6d1277 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:05.634191 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:05.744517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaee426-d1c9-4a7f-9b1f-02762e9651d6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:05.783575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b4bdf-57b4-4332-81b2-15881f6d1277 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:05.785478 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c58dc9-cc6f-4657-a14e-5364e9cf8dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:05.786938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c58dc9-cc6f-4657-a14e-5364e9cf8dc4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:05.786938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c58dc9-cc6f-4657-a14e-5364e9cf8dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:05.786938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c58dc9-cc6f-4657-a14e-5364e9cf8dc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:05.799401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c58dc9-cc6f-4657-a14e-5364e9cf8dc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:05.801138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c58dc9-cc6f-4657-a14e-5364e9cf8dc4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:05.820839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-f5c58dc9-cc6f-4657-a14e-5364e9cf8dc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:05.820839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:05.820839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:05.820839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:05.820839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:05.820839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:05.820839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:05.820839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:05.820839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:05.820839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:05.820839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:05.820839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:05.820839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:05.820839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:05.820839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:05.820839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:05.821951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:05.822999 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:05.822999 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:05.822999 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:05.824539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 245/492] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:05.825967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61811af-2ce4-438e-8b70-ad1a6792f4ee None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:05.826349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61811af-2ce4-438e-8b70-ad1a6792f4ee None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:05.826723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61811af-2ce4-438e-8b70-ad1a6792f4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:05.828663 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e196f-ae22-438c-8455-27ff7a7a434d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:05.829710 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e196f-ae22-438c-8455-27ff7a7a434d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:05.830410 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e196f-ae22-438c-8455-27ff7a7a434d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:05.830833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e196f-ae22-438c-8455-27ff7a7a434d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:05.847450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e196f-ae22-438c-8455-27ff7a7a434d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:05.849620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e196f-ae22-438c-8455-27ff7a7a434d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:05.851769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a61811af-2ce4-438e-8b70-ad1a6792f4ee None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:05.854272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a61811af-2ce4-438e-8b70-ad1a6792f4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:05.861566 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:05.871828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-0b8e196f-ae22-438c-8455-27ff7a7a434d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:05.871828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:05.871828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:05.871828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:05.871828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:05.871828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:05.871828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:05.871828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:05.871828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:05.871828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:05.871828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:05.871828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:05.871828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:05.871828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:05.871828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:05.871828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:05.873384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:05.878150 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:05.878150 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:05.878150 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:05.878150 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 248/493] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:05.878150 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e196f-ae22-438c-8455-27ff7a7a434d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:05.935470 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:05.950421 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 246/494] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:05.951688 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61811af-2ce4-438e-8b70-ad1a6792f4ee None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:06.016045 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e196f-ae22-438c-8455-27ff7a7a434d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:06.070226 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424e56e-96ff-40fe-83ef-5d9bf7aa15ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:06.071126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424e56e-96ff-40fe-83ef-5d9bf7aa15ed None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.071523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424e56e-96ff-40fe-83ef-5d9bf7aa15ed None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.071977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424e56e-96ff-40fe-83ef-5d9bf7aa15ed None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.072899 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424e56e-96ff-40fe-83ef-5d9bf7aa15ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:06.073257 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424e56e-96ff-40fe-83ef-5d9bf7aa15ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:06.074806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61811af-2ce4-438e-8b70-ad1a6792f4ee None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:06.075348 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424e56e-96ff-40fe-83ef-5d9bf7aa15ed None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:06.102242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 249/495] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:06.102242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424e56e-96ff-40fe-83ef-5d9bf7aa15ed None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:06.118540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30bf05-7c82-4b3e-8297-ef59e1ba5517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:06.119196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30bf05-7c82-4b3e-8297-ef59e1ba5517 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.119477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30bf05-7c82-4b3e-8297-ef59e1ba5517 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.119784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30bf05-7c82-4b3e-8297-ef59e1ba5517 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.120447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30bf05-7c82-4b3e-8297-ef59e1ba5517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:06.120755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30bf05-7c82-4b3e-8297-ef59e1ba5517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:06.122185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30bf05-7c82-4b3e-8297-ef59e1ba5517 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:06.137885 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 247/496] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:06.144140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45a30b-d3f1-4790-91b8-e9d57609f8b0 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.144467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45a30b-d3f1-4790-91b8-e9d57609f8b0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.144747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45a30b-d3f1-4790-91b8-e9d57609f8b0 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.146091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 248/497] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:06.146809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45a30b-d3f1-4790-91b8-e9d57609f8b0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:06.187506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:06.214588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:06.240467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424e56e-96ff-40fe-83ef-5d9bf7aa15ed None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:06.272107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45a30b-d3f1-4790-91b8-e9d57609f8b0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:06.277332 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef007b-07b6-45f7-9a1f-a8f82d179034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:06.277967 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef007b-07b6-45f7-9a1f-a8f82d179034 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.278244 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef007b-07b6-45f7-9a1f-a8f82d179034 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.278512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef007b-07b6-45f7-9a1f-a8f82d179034 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.292668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef007b-07b6-45f7-9a1f-a8f82d179034 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:06.298076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-34ef007b-07b6-45f7-9a1f-a8f82d179034 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:06.298076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:06.298076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:06.298076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:06.298076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:06.298076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:06.298076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:06.298076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:06.298076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:06.298076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:06.298076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:06.298076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:06.298076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:06.298076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:06.298076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:06.298076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:06.303003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:06.303003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:06.303003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:06.303003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:06.303003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:06.303003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:06.303003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:06.303003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:06.303003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:06.303003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:06.303003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:06.303003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:06.303003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:06.303003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:06.303003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:06.303003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:06.303942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:06.303942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:06.303942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:06.303942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:06.303942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:06.303942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:06.303942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:06.303942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:06.303942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:06.303942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:06.303942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:06.303942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:06.303942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 250/498] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:06.303942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef007b-07b6-45f7-9a1f-a8f82d179034 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:06.312559 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247cc56-f4c8-48bf-8cda-498dee99bc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:06.313873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247cc56-f4c8-48bf-8cda-498dee99bc78 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.314107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247cc56-f4c8-48bf-8cda-498dee99bc78 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.314376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247cc56-f4c8-48bf-8cda-498dee99bc78 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.327484 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247cc56-f4c8-48bf-8cda-498dee99bc78 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:06.329069 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247cc56-f4c8-48bf-8cda-498dee99bc78 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:06.344846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-6247cc56-f4c8-48bf-8cda-498dee99bc78 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:38:06.344846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:06.344846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:06.344846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:06.344846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:06.344846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:06.344846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:06.344846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:06.344846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:06.344846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:06.344846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:06.344846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:06.344846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:06.344846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:06.344846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:06.344846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:06.345979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:06.347004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:06.347004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:06.347004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:06.347004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:06.347004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:06.347004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:06.347004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:06.347004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:06.347004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:38:06.347004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:06.347004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 249/499] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:06.347678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75b854-928b-41ae-99f9-cb29de697ba6 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.347848 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75b854-928b-41ae-99f9-cb29de697ba6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.348191 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75b854-928b-41ae-99f9-cb29de697ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.367415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-0d75b854-928b-41ae-99f9-cb29de697ba6 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:06.368633 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0d75b854-928b-41ae-99f9-cb29de697ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:06.416796 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef007b-07b6-45f7-9a1f-a8f82d179034 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:06.436164 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:06.438215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 250/500] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:06.439135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75b854-928b-41ae-99f9-cb29de697ba6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:06.456148 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89257ade-32be-49de-9289-edc550521f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:06.458290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89257ade-32be-49de-9289-edc550521f1d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.458290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89257ade-32be-49de-9289-edc550521f1d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.458290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89257ade-32be-49de-9289-edc550521f1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.469081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89257ade-32be-49de-9289-edc550521f1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:06.471550 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89257ade-32be-49de-9289-edc550521f1d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:06.488770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-89257ade-32be-49de-9289-edc550521f1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:06.488770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:06.488770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:06.488770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:06.488770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:06.488770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:06.488770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:06.488770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:06.488770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:06.488770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:06.488770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:06.488770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:06.488770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:06.488770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:06.488770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:06.488770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:06.489632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:06.490489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:06.490489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:06.490489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:06.490489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:06.490489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:06.490489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:06.490489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:06.490489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:06.490489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:06.490489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:06.490489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 251/501] 10.222.0.91 () {56 vars in 1093 bytes} [Thu Sep 8 13:38:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:06.491616 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89257ade-32be-49de-9289-edc550521f1d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:06.492106 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:06.577427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75b854-928b-41ae-99f9-cb29de697ba6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:06.615821 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12febfa6-b9a3-4e59-b8c3-b898b65efc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:06.616458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12febfa6-b9a3-4e59-b8c3-b898b65efc1f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.616759 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12febfa6-b9a3-4e59-b8c3-b898b65efc1f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.617005 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12febfa6-b9a3-4e59-b8c3-b898b65efc1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.627720 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89257ade-32be-49de-9289-edc550521f1d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:06.629123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12febfa6-b9a3-4e59-b8c3-b898b65efc1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:06.630611 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12febfa6-b9a3-4e59-b8c3-b898b65efc1f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:06.646282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-12febfa6-b9a3-4e59-b8c3-b898b65efc1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:06.646282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:06.646282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:06.646282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:06.646282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:06.646282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:06.646282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:06.646282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:06.646282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:06.646282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:06.646282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:06.646282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:06.646282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:06.646282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:06.646282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:06.646282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:06.647298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:06.648328 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:06.648328 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:06.648328 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:06.652349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 251/502] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:06.653207 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12febfa6-b9a3-4e59-b8c3-b898b65efc1f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:06.668766 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ba10d-3815-4555-977c-eca0c9de5983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:06.669751 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ba10d-3815-4555-977c-eca0c9de5983 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.670216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ba10d-3815-4555-977c-eca0c9de5983 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.670687 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ba10d-3815-4555-977c-eca0c9de5983 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.671516 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ba10d-3815-4555-977c-eca0c9de5983 None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:06.672029 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ba10d-3815-4555-977c-eca0c9de5983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:06.673742 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ba10d-3815-4555-977c-eca0c9de5983 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:06.683126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 252/503] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:06.684324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ba10d-3815-4555-977c-eca0c9de5983 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:06.731309 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:06.755128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:06.764788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12febfa6-b9a3-4e59-b8c3-b898b65efc1f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:06.795368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ba10d-3815-4555-977c-eca0c9de5983 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:06.801732 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442ae7d-a48f-4b12-a14c-30234f3e8708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:06.802382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442ae7d-a48f-4b12-a14c-30234f3e8708 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.802663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442ae7d-a48f-4b12-a14c-30234f3e8708 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.802973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442ae7d-a48f-4b12-a14c-30234f3e8708 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.803740 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442ae7d-a48f-4b12-a14c-30234f3e8708 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:06.804048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442ae7d-a48f-4b12-a14c-30234f3e8708 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:06.805604 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442ae7d-a48f-4b12-a14c-30234f3e8708 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:06.835590 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73811d2e-76d2-4abf-9eba-cf85c0550850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:06.836113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73811d2e-76d2-4abf-9eba-cf85c0550850 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.836390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73811d2e-76d2-4abf-9eba-cf85c0550850 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.836661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73811d2e-76d2-4abf-9eba-cf85c0550850 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.837394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73811d2e-76d2-4abf-9eba-cf85c0550850 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:06.837699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73811d2e-76d2-4abf-9eba-cf85c0550850 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:06.839171 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73811d2e-76d2-4abf-9eba-cf85c0550850 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:06.854633 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 252/504] 10.222.0.91 () {56 vars in 1137 bytes} [Thu Sep 8 13:38:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:06.855538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442ae7d-a48f-4b12-a14c-30234f3e8708 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:06.882391 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 253/505] 10.222.0.91 () {56 vars in 1133 bytes} [Thu Sep 8 13:38:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:06.883265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73811d2e-76d2-4abf-9eba-cf85c0550850 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:06.962322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442ae7d-a48f-4b12-a14c-30234f3e8708 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:06.983060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:06.988584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73811d2e-76d2-4abf-9eba-cf85c0550850 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.002101 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eadb90-905c-4774-9cd1-8911f862247d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.002717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eadb90-905c-4774-9cd1-8911f862247d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.003333 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eadb90-905c-4774-9cd1-8911f862247d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.003333 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eadb90-905c-4774-9cd1-8911f862247d None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.003995 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eadb90-905c-4774-9cd1-8911f862247d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:07.004296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eadb90-905c-4774-9cd1-8911f862247d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:07.005851 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eadb90-905c-4774-9cd1-8911f862247d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:07.020600 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 253/506] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:07.021452 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eadb90-905c-4774-9cd1-8911f862247d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.029953 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa39d0-d791-4bbe-8d5c-8f4059a3f6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.030923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa39d0-d791-4bbe-8d5c-8f4059a3f6ca None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.031256 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa39d0-d791-4bbe-8d5c-8f4059a3f6ca None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.031965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa39d0-d791-4bbe-8d5c-8f4059a3f6ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.033368 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:07.043870 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa39d0-d791-4bbe-8d5c-8f4059a3f6ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:07.045350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa39d0-d791-4bbe-8d5c-8f4059a3f6ca None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:07.062198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-14aa39d0-d791-4bbe-8d5c-8f4059a3f6ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:07.062198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:07.062198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:07.062198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:07.062198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:07.062198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:07.062198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:07.062198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:07.062198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:07.062198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:07.062198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:07.062198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:07.062198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:07.062198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:07.062198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:07.062198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:07.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:07.064720 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:07.064720 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:07.064720 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:07.064720 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 254/507] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:07.064970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa39d0-d791-4bbe-8d5c-8f4059a3f6ca None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.132745 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eadb90-905c-4774-9cd1-8911f862247d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.170600 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f8969-f2a5-4ed8-b4cd-babc651d7624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.171259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f8969-f2a5-4ed8-b4cd-babc651d7624 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.171532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f8969-f2a5-4ed8-b4cd-babc651d7624 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.171798 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f8969-f2a5-4ed8-b4cd-babc651d7624 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.174510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa39d0-d791-4bbe-8d5c-8f4059a3f6ca None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.183690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f8969-f2a5-4ed8-b4cd-babc651d7624 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:07.189298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-1d0f8969-f2a5-4ed8-b4cd-babc651d7624 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:07.189298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:07.189298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:07.189298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:07.189298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:07.189298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:07.189298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:07.189298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:07.189298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:07.189298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:07.189298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:07.189298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:07.189298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:07.189298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:07.189298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:07.189298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:07.190420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:07.190420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:07.190420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:07.190420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:07.190420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:07.190420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:07.190420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:07.190420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:07.190420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:07.190420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:07.190420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:07.190420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:07.190420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:07.190420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:07.190420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:07.190420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:07.191291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:07.191291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:07.191291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:07.191291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:07.191291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:07.191291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:07.191291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:07.191291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:07.191291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:07.191291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:07.191291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:07.191291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:07.191291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 254/508] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:07.191936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f8969-f2a5-4ed8-b4cd-babc651d7624 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.222545 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c551a-7c14-4711-abd5-44b3c628d619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.223517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c551a-7c14-4711-abd5-44b3c628d619 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.223892 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c551a-7c14-4711-abd5-44b3c628d619 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.224316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c551a-7c14-4711-abd5-44b3c628d619 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.252293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:07.266730 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:07.303014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f8969-f2a5-4ed8-b4cd-babc651d7624 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.307189 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c551a-7c14-4711-abd5-44b3c628d619 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:07.308749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c551a-7c14-4711-abd5-44b3c628d619 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:07.337861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcaf21f-2433-419d-8179-7723141958bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.338462 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaf21f-2433-419d-8179-7723141958bf None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.338733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaf21f-2433-419d-8179-7723141958bf None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.339241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaf21f-2433-419d-8179-7723141958bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.413884 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 255/509] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:07.414679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c551a-7c14-4711-abd5-44b3c628d619 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.416678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcaf21f-2433-419d-8179-7723141958bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:07.418409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcaf21f-2433-419d-8179-7723141958bf None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:07.473347 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:07.496809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:07.533452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c551a-7c14-4711-abd5-44b3c628d619 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.534189 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 255/510] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:07.535057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcaf21f-2433-419d-8179-7723141958bf None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.573895 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb3024-9d49-4917-b6ac-70c0e2fe6344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.574611 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb3024-9d49-4917-b6ac-70c0e2fe6344 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.574913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb3024-9d49-4917-b6ac-70c0e2fe6344 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.575181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb3024-9d49-4917-b6ac-70c0e2fe6344 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.575825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb3024-9d49-4917-b6ac-70c0e2fe6344 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:07.577258 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb3024-9d49-4917-b6ac-70c0e2fe6344 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:07.636216 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcaf21f-2433-419d-8179-7723141958bf None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.670318 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d643b1f-75e7-4688-9465-e41af6f0af7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.670842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d643b1f-75e7-4688-9465-e41af6f0af7e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.671139 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d643b1f-75e7-4688-9465-e41af6f0af7e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.671412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d643b1f-75e7-4688-9465-e41af6f0af7e None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.672036 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d643b1f-75e7-4688-9465-e41af6f0af7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:07.672335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d643b1f-75e7-4688-9465-e41af6f0af7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:07.674247 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d643b1f-75e7-4688-9465-e41af6f0af7e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:07.681880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 256/511] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:07 2022] POST /identity/v3/users => generated 247 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:07.690468 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb3024-9d49-4917-b6ac-70c0e2fe6344 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.696476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 256/512] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:07.697815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d643b1f-75e7-4688-9465-e41af6f0af7e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.733293 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:07.817642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb3024-9d49-4917-b6ac-70c0e2fe6344 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.822155 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d643b1f-75e7-4688-9465-e41af6f0af7e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.826476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:07.861307 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2cad1-f5c8-44a0-a692-f992a6e42761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.861932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2cad1-f5c8-44a0-a692-f992a6e42761 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.862213 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2cad1-f5c8-44a0-a692-f992a6e42761 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.862488 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2cad1-f5c8-44a0-a692-f992a6e42761 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.863189 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e2cad1-f5c8-44a0-a692-f992a6e42761 None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:07.863498 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e2cad1-f5c8-44a0-a692-f992a6e42761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:07.864995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e2cad1-f5c8-44a0-a692-f992a6e42761 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:07.869983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec9187-6526-406c-9e41-3c23afefe75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.870928 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec9187-6526-406c-9e41-3c23afefe75c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.871325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec9187-6526-406c-9e41-3c23afefe75c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.872151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec9187-6526-406c-9e41-3c23afefe75c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.876652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 257/513] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:07.882983 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2cad1-f5c8-44a0-a692-f992a6e42761 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.888246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec9187-6526-406c-9e41-3c23afefe75c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:07.889738 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec9187-6526-406c-9e41-3c23afefe75c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:07.906634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-4bec9187-6526-406c-9e41-3c23afefe75c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:07.906634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:07.906634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:07.906634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:07.906634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:07.906634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:07.906634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:07.906634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:07.906634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:07.906634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:07.906634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:07.906634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:07.906634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:07.906634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:07.906634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:07.906634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:07.908320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:07.914978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:07.914978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:07.914978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:07.914978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 257/514] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:07.914978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec9187-6526-406c-9e41-3c23afefe75c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.996467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2cad1-f5c8-44a0-a692-f992a6e42761 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.006707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:08.031694 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec9187-6526-406c-9e41-3c23afefe75c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.035512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6934a1f-2f70-4403-9ffb-c266862bbdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:08.036140 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6934a1f-2f70-4403-9ffb-c266862bbdb5 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:08.036410 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6934a1f-2f70-4403-9ffb-c266862bbdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:08.036721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6934a1f-2f70-4403-9ffb-c266862bbdb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:08.049171 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6934a1f-2f70-4403-9ffb-c266862bbdb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:08.050763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6934a1f-2f70-4403-9ffb-c266862bbdb5 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:08.068324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-a6934a1f-2f70-4403-9ffb-c266862bbdb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:08.068324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:08.068324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:08.068324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:08.068324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:08.068324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:08.068324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:08.068324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:08.068324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:08.068324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:08.068324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:08.068324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:08.068324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:08.068324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:08.068324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:08.068324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:08.069263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:08.070086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:08.070086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:08.070086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:08.070086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 258/515] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:08.070658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6934a1f-2f70-4403-9ffb-c266862bbdb5 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:08.077321 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814a7b5-fee0-41f4-af5f-732b2dc3671a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:08.077973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814a7b5-fee0-41f4-af5f-732b2dc3671a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:08.078253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814a7b5-fee0-41f4-af5f-732b2dc3671a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:08.078527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814a7b5-fee0-41f4-af5f-732b2dc3671a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:08.088137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814a7b5-fee0-41f4-af5f-732b2dc3671a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:08.089636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814a7b5-fee0-41f4-af5f-732b2dc3671a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:08.097964 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:08.104304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-7814a7b5-fee0-41f4-af5f-732b2dc3671a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:38:08.104304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:08.104304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:08.104304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:08.104304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:08.104304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:08.104304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:08.104304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:08.104304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:08.104304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:08.104304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:08.104304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:08.104304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:08.104304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:08.104304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:08.104304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:08.105282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:08.106183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:08.106183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:08.106183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:08.106183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:08.106183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:08.106183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:08.106183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:08.106183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:08.106183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:38:08.106183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:08.106183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 258/516] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:08.106728 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814a7b5-fee0-41f4-af5f-732b2dc3671a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:08.205782 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6934a1f-2f70-4403-9ffb-c266862bbdb5 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.240566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814a7b5-fee0-41f4-af5f-732b2dc3671a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.256130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0109d1-924b-44bc-a008-5dc8a3a0d3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:08.257043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0109d1-924b-44bc-a008-5dc8a3a0d3c9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:08.257043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0109d1-924b-44bc-a008-5dc8a3a0d3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:08.257208 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0109d1-924b-44bc-a008-5dc8a3a0d3c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:08.269523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0109d1-924b-44bc-a008-5dc8a3a0d3c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:08.271061 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0109d1-924b-44bc-a008-5dc8a3a0d3c9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:08.280427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c2ea5-c663-4463-bff1-cc5aca94ca99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:08.281052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c2ea5-c663-4463-bff1-cc5aca94ca99 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:08.281332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c2ea5-c663-4463-bff1-cc5aca94ca99 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:08.281612 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c2ea5-c663-4463-bff1-cc5aca94ca99 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:08.282305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c2ea5-c663-4463-bff1-cc5aca94ca99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:08.282609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c2ea5-c663-4463-bff1-cc5aca94ca99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:08.284076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c2ea5-c663-4463-bff1-cc5aca94ca99 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:08.289129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-0b0109d1-924b-44bc-a008-5dc8a3a0d3c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:08.289129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:08.289129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:08.289129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:08.289129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:08.289129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:08.289129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:08.289129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:08.289129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:08.289129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:08.289129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:08.289129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:08.289129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:08.289129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:08.289129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:08.289129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:08.290115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:08.297399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:08.297399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:08.297399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:08.297399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 259/517] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:08.297399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0109d1-924b-44bc-a008-5dc8a3a0d3c9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:08.302345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:08.306073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 259/518] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:08.311318 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c2ea5-c663-4463-bff1-cc5aca94ca99 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:08.390818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:08.435685 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c2ea5-c663-4463-bff1-cc5aca94ca99 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.459341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0109d1-924b-44bc-a008-5dc8a3a0d3c9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.478623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6798bb2-d420-478c-aaae-dbe742309bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:08.479402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6798bb2-d420-478c-aaae-dbe742309bfa None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:08.479685 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6798bb2-d420-478c-aaae-dbe742309bfa None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:08.479959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6798bb2-d420-478c-aaae-dbe742309bfa None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:08.480718 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6798bb2-d420-478c-aaae-dbe742309bfa None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:08.481319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6798bb2-d420-478c-aaae-dbe742309bfa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:08.482499 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6798bb2-d420-478c-aaae-dbe742309bfa None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:08.507654 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 260/519] 10.222.0.91 () {56 vars in 1137 bytes} [Thu Sep 8 13:38:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:08.507654 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6798bb2-d420-478c-aaae-dbe742309bfa None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:08.517635 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd358fc-8994-49b0-b860-b64b5c0a7562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:08.518309 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd358fc-8994-49b0-b860-b64b5c0a7562 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:08.518582 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd358fc-8994-49b0-b860-b64b5c0a7562 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:08.518944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd358fc-8994-49b0-b860-b64b5c0a7562 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:08.519576 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd358fc-8994-49b0-b860-b64b5c0a7562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:08.519885 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd358fc-8994-49b0-b860-b64b5c0a7562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:08.521307 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd358fc-8994-49b0-b860-b64b5c0a7562 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:08.540044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 260/520] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:08.540044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd358fc-8994-49b0-b860-b64b5c0a7562 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:08.577082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:08.623048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6798bb2-d420-478c-aaae-dbe742309bfa None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.660079 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452e6b6-9508-401a-8b8e-551d56ee633a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:08.660743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452e6b6-9508-401a-8b8e-551d56ee633a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:08.660961 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452e6b6-9508-401a-8b8e-551d56ee633a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:08.661254 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452e6b6-9508-401a-8b8e-551d56ee633a None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:08.661895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452e6b6-9508-401a-8b8e-551d56ee633a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:08.662198 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452e6b6-9508-401a-8b8e-551d56ee633a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:08.663628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452e6b6-9508-401a-8b8e-551d56ee633a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:08.670726 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd358fc-8994-49b0-b860-b64b5c0a7562 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.681061 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 261/521] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:08.681061 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452e6b6-9508-401a-8b8e-551d56ee633a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:08.715096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:08.722625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9889e2-900c-46e5-85ba-a1a71e044283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:08.722625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9889e2-900c-46e5-85ba-a1a71e044283 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:08.727811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9889e2-900c-46e5-85ba-a1a71e044283 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:08.727811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9889e2-900c-46e5-85ba-a1a71e044283 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:08.731683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9889e2-900c-46e5-85ba-a1a71e044283 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:08.733268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9889e2-900c-46e5-85ba-a1a71e044283 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:08.747246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-2e9889e2-900c-46e5-85ba-a1a71e044283 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:08.747246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:08.747246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:08.747246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:08.747246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:08.747246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:08.747246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:08.747246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:08.747246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:08.747246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:08.747246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:08.747246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:08.747246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:08.747246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:08.747246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:08.748294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:08.749167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:08.749167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:08.749167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:08.749167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:08.749167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:08.749167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:08.749167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:08.749167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:08.749167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 261/522] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:08.749619 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9889e2-900c-46e5-85ba-a1a71e044283 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:08.801077 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452e6b6-9508-401a-8b8e-551d56ee633a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.820482 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:08.840749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c0c73-ff28-4dba-9765-383db226d510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:08.841477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c0c73-ff28-4dba-9765-383db226d510 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:08.841754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c0c73-ff28-4dba-9765-383db226d510 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:08.842049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c0c73-ff28-4dba-9765-383db226d510 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:08.872922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9889e2-900c-46e5-85ba-a1a71e044283 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.910658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c343f7-3d04-4a3e-8f0e-bcb4924d1eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:08.911452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c343f7-3d04-4a3e-8f0e-bcb4924d1eff None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:08.911744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c343f7-3d04-4a3e-8f0e-bcb4924d1eff None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:08.912099 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c343f7-3d04-4a3e-8f0e-bcb4924d1eff None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:08.920462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c343f7-3d04-4a3e-8f0e-bcb4924d1eff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:08.921935 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c343f7-3d04-4a3e-8f0e-bcb4924d1eff None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:08.924043 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029c0c73-ff28-4dba-9765-383db226d510 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:08.925566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029c0c73-ff28-4dba-9765-383db226d510 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:08.934866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-d5c343f7-3d04-4a3e-8f0e-bcb4924d1eff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:08.934866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:08.934866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:08.934866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:08.934866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:08.934866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:08.934866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:08.934866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:08.934866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:08.934866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:08.934866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:08.934866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:08.934866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:08.934866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:08.934866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:08.935889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:08.936788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:08.936788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:08.936788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:08.936788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:08.936788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:08.936788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:08.936788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:08.936788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:08.936788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 262/523] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:08.937367 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c343f7-3d04-4a3e-8f0e-bcb4924d1eff None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:08.996237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:09.052760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c343f7-3d04-4a3e-8f0e-bcb4924d1eff None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:09.057570 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 262/524] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:09.057570 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c0c73-ff28-4dba-9765-383db226d510 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:09.075361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:09.099080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c0659-ca27-4f5f-956f-9b999df3179a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:09.099703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c0659-ca27-4f5f-956f-9b999df3179a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:09.099981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c0659-ca27-4f5f-956f-9b999df3179a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:09.100251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c0659-ca27-4f5f-956f-9b999df3179a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:09.108086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c0659-ca27-4f5f-956f-9b999df3179a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:09.109581 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c0659-ca27-4f5f-956f-9b999df3179a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:09.121404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-6a1c0659-ca27-4f5f-956f-9b999df3179a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:38:09.121404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:09.121404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:09.121404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:09.121404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:09.121404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:09.121404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:09.121404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:09.121404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:09.121404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:09.121404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:09.121404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:09.121404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:09.121404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:09.121404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:09.121404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:09.122605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:09.123807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:09.123807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:09.123807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:09.123807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:09.123807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:09.123807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:09.123807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:09.123807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:09.123807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:38:09.123807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:09.127936 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 263/525] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:09.128554 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c0659-ca27-4f5f-956f-9b999df3179a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:09.171299 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c0c73-ff28-4dba-9765-383db226d510 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:09.209639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371fef7-9e0c-4874-8846-5ebbe1f3254e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:09.210539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371fef7-9e0c-4874-8846-5ebbe1f3254e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:09.210874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371fef7-9e0c-4874-8846-5ebbe1f3254e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:09.211260 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371fef7-9e0c-4874-8846-5ebbe1f3254e None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:09.212006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371fef7-9e0c-4874-8846-5ebbe1f3254e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:09.212367 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371fef7-9e0c-4874-8846-5ebbe1f3254e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:09.213964 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371fef7-9e0c-4874-8846-5ebbe1f3254e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:09.238335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 263/526] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:09.239266 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371fef7-9e0c-4874-8846-5ebbe1f3254e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:09.242589 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c0659-ca27-4f5f-956f-9b999df3179a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:09.256409 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:09.279644 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23fe18-93b5-4bd8-8f2b-2bbdb4527198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:09.280337 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23fe18-93b5-4bd8-8f2b-2bbdb4527198 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:09.280685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23fe18-93b5-4bd8-8f2b-2bbdb4527198 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:09.281027 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23fe18-93b5-4bd8-8f2b-2bbdb4527198 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:09.281786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23fe18-93b5-4bd8-8f2b-2bbdb4527198 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:09.282187 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23fe18-93b5-4bd8-8f2b-2bbdb4527198 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:09.283728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23fe18-93b5-4bd8-8f2b-2bbdb4527198 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:09.303714 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 264/527] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:09.304659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23fe18-93b5-4bd8-8f2b-2bbdb4527198 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:09.333348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:09.357726 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371fef7-9e0c-4874-8846-5ebbe1f3254e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:09.395588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3619701-c458-47d3-813a-8db1598f96a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:09.396226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3619701-c458-47d3-813a-8db1598f96a7 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:09.396507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3619701-c458-47d3-813a-8db1598f96a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:09.396798 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3619701-c458-47d3-813a-8db1598f96a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:09.409533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3619701-c458-47d3-813a-8db1598f96a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:09.411206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3619701-c458-47d3-813a-8db1598f96a7 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:09.426957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-c3619701-c458-47d3-813a-8db1598f96a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:09.426957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:09.426957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:09.426957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:09.426957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:09.426957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:09.426957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:09.426957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:09.426957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:09.426957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:09.426957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:09.426957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:09.426957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:09.426957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:09.426957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:09.426957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:09.428076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:09.429112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:09.429112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:09.429112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:09.429323 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23fe18-93b5-4bd8-8f2b-2bbdb4527198 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:09.430269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 264/528] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:09.435557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3619701-c458-47d3-813a-8db1598f96a7 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:09.470476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77f31c-2be2-4543-9682-3c1949cb734e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:09.471207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77f31c-2be2-4543-9682-3c1949cb734e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:09.471640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77f31c-2be2-4543-9682-3c1949cb734e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:09.471944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77f31c-2be2-4543-9682-3c1949cb734e None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:09.472744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77f31c-2be2-4543-9682-3c1949cb734e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:09.473120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77f31c-2be2-4543-9682-3c1949cb734e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:09.474624 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77f31c-2be2-4543-9682-3c1949cb734e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:09.498500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 265/529] 10.222.0.91 () {56 vars in 1139 bytes} [Thu Sep 8 13:38:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:09.500602 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77f31c-2be2-4543-9682-3c1949cb734e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:09.539514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:09.565710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3619701-c458-47d3-813a-8db1598f96a7 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:09.601206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:09.609118 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf08c8-73b4-4c44-ae85-b0dd5cafd607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:09.609744 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf08c8-73b4-4c44-ae85-b0dd5cafd607 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:09.610497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf08c8-73b4-4c44-ae85-b0dd5cafd607 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:09.610497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf08c8-73b4-4c44-ae85-b0dd5cafd607 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:09.611116 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf08c8-73b4-4c44-ae85-b0dd5cafd607 None admin] RBAC: Adding query filter params (role.id=44e187abae1d4b248fd145ad9cb97ed2, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=01b060c70e1b440186eeac37d9c34567) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:09.613873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf08c8-73b4-4c44-ae85-b0dd5cafd607 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:09.613873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf08c8-73b4-4c44-ae85-b0dd5cafd607 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:09.619699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 265/530] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:09 2022] GET /identity/v3/role_assignments?user.id=01b060c70e1b440186eeac37d9c34567&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=44e187abae1d4b248fd145ad9cb97ed2 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:09.626831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77f31c-2be2-4543-9682-3c1949cb734e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:09.627850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf08c8-73b4-4c44-ae85-b0dd5cafd607 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:09.689812 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a10d0-6f4a-4ae7-a63f-0fefa7f7dd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:09.690495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a10d0-6f4a-4ae7-a63f-0fefa7f7dd59 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:09.690786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a10d0-6f4a-4ae7-a63f-0fefa7f7dd59 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:09.691297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a10d0-6f4a-4ae7-a63f-0fefa7f7dd59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:09.692076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a10d0-6f4a-4ae7-a63f-0fefa7f7dd59 None admin] RBAC: Adding query filter params (role.id=44e187abae1d4b248fd145ad9cb97ed2, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=38127f41852d412ea6ad270f1618fce2) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:09.692389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a10d0-6f4a-4ae7-a63f-0fefa7f7dd59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:09.693928 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a10d0-6f4a-4ae7-a63f-0fefa7f7dd59 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:09.707581 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 266/531] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:09 2022] GET /identity/v3/role_assignments?user.id=38127f41852d412ea6ad270f1618fce2&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=44e187abae1d4b248fd145ad9cb97ed2 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:09.709096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a10d0-6f4a-4ae7-a63f-0fefa7f7dd59 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:09.814991 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf08c8-73b4-4c44-ae85-b0dd5cafd607 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:09.875720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ab2c4-8814-4d0a-bd8f-cfdbc504899e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:09.877162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ab2c4-8814-4d0a-bd8f-cfdbc504899e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:09.877162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ab2c4-8814-4d0a-bd8f-cfdbc504899e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:09.877162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ab2c4-8814-4d0a-bd8f-cfdbc504899e None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:09.877616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ab2c4-8814-4d0a-bd8f-cfdbc504899e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:09.878857 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ab2c4-8814-4d0a-bd8f-cfdbc504899e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:09.879815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ab2c4-8814-4d0a-bd8f-cfdbc504899e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:09.907721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 266/532] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:09.916687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ab2c4-8814-4d0a-bd8f-cfdbc504899e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:09.973468 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:09.984561 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a10d0-6f4a-4ae7-a63f-0fefa7f7dd59 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:10.026580 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:10.041953 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cb24c-f20d-4074-aded-b62427be009c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:10.043985 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cb24c-f20d-4074-aded-b62427be009c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:10.044460 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cb24c-f20d-4074-aded-b62427be009c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:10.045113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cb24c-f20d-4074-aded-b62427be009c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:10.081896 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ab2c4-8814-4d0a-bd8f-cfdbc504899e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:10.121412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53cf715-6cdf-43e8-86a3-170b5e37c253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:10.122295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cf715-6cdf-43e8-86a3-170b5e37c253 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:10.122704 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cf715-6cdf-43e8-86a3-170b5e37c253 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:10.123273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53cf715-6cdf-43e8-86a3-170b5e37c253 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:10.131201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cf715-6cdf-43e8-86a3-170b5e37c253 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:10.132945 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53cf715-6cdf-43e8-86a3-170b5e37c253 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:10.144735 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cb24c-f20d-4074-aded-b62427be009c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:10.146417 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cb24c-f20d-4074-aded-b62427be009c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:10.146671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-d53cf715-6cdf-43e8-86a3-170b5e37c253 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:10.146671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:10.146671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:10.146671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:10.146671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:10.146671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:10.146671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:10.146671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:10.146671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:10.146671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:10.146671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:10.146671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:10.146671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:10.146671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:10.146671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:10.147652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:10.148758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:10.148758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:10.148758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:10.148758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:10.148758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:10.148758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:10.148758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:10.148758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:10.151839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 267/533] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:10.153474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae0c0e-c23b-4cd9-97e9-7272007e3602 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:10.153855 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae0c0e-c23b-4cd9-97e9-7272007e3602 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:10.154222 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae0c0e-c23b-4cd9-97e9-7272007e3602 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:10.160782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 268/534] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:10.163055 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf460a7-2040-4c32-ab5c-62ed6ef7a39e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:10.163339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf460a7-2040-4c32-ab5c-62ed6ef7a39e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:10.163609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf460a7-2040-4c32-ab5c-62ed6ef7a39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:10.183136 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-9cf460a7-2040-4c32-ab5c-62ed6ef7a39e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:10.184338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9cf460a7-2040-4c32-ab5c-62ed6ef7a39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:10.274932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 269/535] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:10.274932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf460a7-2040-4c32-ab5c-62ed6ef7a39e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:10.286930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:10.305443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 267/536] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:10.311339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cb24c-f20d-4074-aded-b62427be009c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:10.338922 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:10.450736 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf460a7-2040-4c32-ab5c-62ed6ef7a39e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:10.452302 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cb24c-f20d-4074-aded-b62427be009c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:10.501705 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee9945-f96a-4c52-b686-782e2b86d94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:10.502915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee9945-f96a-4c52-b686-782e2b86d94e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:10.503498 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee9945-f96a-4c52-b686-782e2b86d94e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:10.504074 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee9945-f96a-4c52-b686-782e2b86d94e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:10.519145 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee9945-f96a-4c52-b686-782e2b86d94e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:10.520788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee9945-f96a-4c52-b686-782e2b86d94e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:10.537971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-acee9945-f96a-4c52-b686-782e2b86d94e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:10.537971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:10.537971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:10.537971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:10.537971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:10.537971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:10.537971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:10.537971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:10.537971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:10.537971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:10.537971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:10.537971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:10.537971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:10.537971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:10.537971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:10.537971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:10.538976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:10.539820 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:10.539820 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:10.539820 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:10.539820 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 268/537] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:10.545051 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee9945-f96a-4c52-b686-782e2b86d94e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:10.562545 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9861f-d826-4bc6-9c94-6d36361440a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:10.563207 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9861f-d826-4bc6-9c94-6d36361440a3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:10.565864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9861f-d826-4bc6-9c94-6d36361440a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:10.566179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9861f-d826-4bc6-9c94-6d36361440a3 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:10.567005 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9861f-d826-4bc6-9c94-6d36361440a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:10.567318 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9861f-d826-4bc6-9c94-6d36361440a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:10.568809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9861f-d826-4bc6-9c94-6d36361440a3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:10.599329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 270/538] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:10.605621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9861f-d826-4bc6-9c94-6d36361440a3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:10.660626 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:10.749085 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee9945-f96a-4c52-b686-782e2b86d94e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:10.755509 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9861f-d826-4bc6-9c94-6d36361440a3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:10.759515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:10.795273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4990d0-c0f9-4dfe-bd12-5784e415228f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:10.795883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4990d0-c0f9-4dfe-bd12-5784e415228f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:10.796266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4990d0-c0f9-4dfe-bd12-5784e415228f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:10.796630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4990d0-c0f9-4dfe-bd12-5784e415228f None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:10.797380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4990d0-c0f9-4dfe-bd12-5784e415228f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:10.798199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4990d0-c0f9-4dfe-bd12-5784e415228f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:10.799736 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4990d0-c0f9-4dfe-bd12-5784e415228f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:10.803410 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a27421-0ab3-4446-8a43-5f8dde6e650b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:10.804107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a27421-0ab3-4446-8a43-5f8dde6e650b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:10.804453 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a27421-0ab3-4446-8a43-5f8dde6e650b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:10.804788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a27421-0ab3-4446-8a43-5f8dde6e650b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:10.805659 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a27421-0ab3-4446-8a43-5f8dde6e650b None admin] RBAC: Adding query filter params (role.id=44e187abae1d4b248fd145ad9cb97ed2, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=ca225b99584e40f4a4065bf10264974d) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:10.806103 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a27421-0ab3-4446-8a43-5f8dde6e650b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:10.807738 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a27421-0ab3-4446-8a43-5f8dde6e650b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:10.817814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 271/540] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:10 2022] GET /identity/v3/role_assignments?user.id=ca225b99584e40f4a4065bf10264974d&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=44e187abae1d4b248fd145ad9cb97ed2 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:10.822926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a27421-0ab3-4446-8a43-5f8dde6e650b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:10.823122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 269/540] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:10.828282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4990d0-c0f9-4dfe-bd12-5784e415228f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:11.019138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a27421-0ab3-4446-8a43-5f8dde6e650b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:11.036232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4990d0-c0f9-4dfe-bd12-5784e415228f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:11.047224 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:11.099543 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31bf9d-7c22-4985-8765-dc6eea926a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:11.100181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31bf9d-7c22-4985-8765-dc6eea926a67 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:11.100457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31bf9d-7c22-4985-8765-dc6eea926a67 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:11.101294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31bf9d-7c22-4985-8765-dc6eea926a67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:11.102432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a8122-eed0-4405-affe-15ef1cb466b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:11.103083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a8122-eed0-4405-affe-15ef1cb466b6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:11.103357 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a8122-eed0-4405-affe-15ef1cb466b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:11.103621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a8122-eed0-4405-affe-15ef1cb466b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:11.118428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31bf9d-7c22-4985-8765-dc6eea926a67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:11.120216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31bf9d-7c22-4985-8765-dc6eea926a67 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:11.129938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a8122-eed0-4405-affe-15ef1cb466b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:11.135126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-7e8a8122-eed0-4405-affe-15ef1cb466b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:11.135126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:11.135126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:11.135126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:11.135126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:11.135126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:11.135126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:11.135126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:11.135126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:11.135126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:11.135126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:11.135126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:11.135126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:11.135126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:11.135126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:11.135126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:11.136025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:11.136025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:11.136025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:11.136025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:11.136025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:11.136025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:11.136025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:11.136025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:11.136025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:11.136025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:11.136025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:11.136025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:11.136025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:11.136025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:11.136025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:11.136025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:11.136803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:11.136803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:11.136803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:11.136803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:11.136803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:11.136803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:11.136803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:11.136803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:11.136803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:11.136803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:11.136803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:11.136803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:11.141181 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 272/541] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:11.142132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a8122-eed0-4405-affe-15ef1cb466b6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:11.145423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-6a31bf9d-7c22-4985-8765-dc6eea926a67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:11.145423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:11.145423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:11.145423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:11.145423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:11.145423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:11.145423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:11.145423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:11.145423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:11.145423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:11.145423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:11.145423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:11.145423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:11.145423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:11.145423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:11.146263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:11.147146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:11.147146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:11.147146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:11.147146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:11.147146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:11.147146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:11.147146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:11.147146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:11.151070 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 270/542] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:11.168130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31bf9d-7c22-4985-8765-dc6eea926a67 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:11.200373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:11.333180 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a8122-eed0-4405-affe-15ef1cb466b6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:11.368300 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31bf9d-7c22-4985-8765-dc6eea926a67 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:11.410583 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9a918-c797-42dd-ab67-2f5700dbf226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:11.411228 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9a918-c797-42dd-ab67-2f5700dbf226 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:11.411508 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9a918-c797-42dd-ab67-2f5700dbf226 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:11.411776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9a918-c797-42dd-ab67-2f5700dbf226 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:11.412491 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9a918-c797-42dd-ab67-2f5700dbf226 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:11.412797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9a918-c797-42dd-ab67-2f5700dbf226 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:11.414247 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9a918-c797-42dd-ab67-2f5700dbf226 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:11.418678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298442f-a8ad-4fa1-9641-2be9ed014784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:11.419325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298442f-a8ad-4fa1-9641-2be9ed014784 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:11.419601 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298442f-a8ad-4fa1-9641-2be9ed014784 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:11.419873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298442f-a8ad-4fa1-9641-2be9ed014784 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:11.420540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298442f-a8ad-4fa1-9641-2be9ed014784 None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:11.420846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298442f-a8ad-4fa1-9641-2be9ed014784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:11.422239 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e298442f-a8ad-4fa1-9641-2be9ed014784 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:11.440089 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 273/543] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:11.442554 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 271/544] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:11.448120 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298442f-a8ad-4fa1-9641-2be9ed014784 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:11.450022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9a918-c797-42dd-ab67-2f5700dbf226 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:11.505995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:11.637597 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:11.649733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298442f-a8ad-4fa1-9641-2be9ed014784 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:11.678141 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9a918-c797-42dd-ab67-2f5700dbf226 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:11.688794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d60338-0b02-4f5a-8a89-848d7baca1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:11.689734 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d60338-0b02-4f5a-8a89-848d7baca1aa None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:11.690221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d60338-0b02-4f5a-8a89-848d7baca1aa None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:11.690672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d60338-0b02-4f5a-8a89-848d7baca1aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:11.702182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d60338-0b02-4f5a-8a89-848d7baca1aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:11.704947 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d60338-0b02-4f5a-8a89-848d7baca1aa None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:11.721231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-d6d60338-0b02-4f5a-8a89-848d7baca1aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:11.721231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:11.721231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:11.721231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:11.721231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:11.721231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:11.721231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:11.721231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:11.721231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:11.721231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:11.721231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:11.721231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:11.721231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:11.721231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:11.721231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:11.721231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:11.722201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:11.723408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:11.723408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:11.723408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:11.725210 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 274/545] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:11.733131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d60338-0b02-4f5a-8a89-848d7baca1aa None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:11.761411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532703a-2e5f-43f7-831c-049916c1f3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:11.762165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532703a-2e5f-43f7-831c-049916c1f3a6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:11.762442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532703a-2e5f-43f7-831c-049916c1f3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:11.762709 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532703a-2e5f-43f7-831c-049916c1f3a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:11.763535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532703a-2e5f-43f7-831c-049916c1f3a6 None admin] RBAC: Adding query filter params (role.id=44e187abae1d4b248fd145ad9cb97ed2, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=cdbbaa64ad4844a0b2698dee82f95f23) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:11.764074 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532703a-2e5f-43f7-831c-049916c1f3a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:11.767911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532703a-2e5f-43f7-831c-049916c1f3a6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:11.779679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 272/546] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:11 2022] GET /identity/v3/role_assignments?user.id=cdbbaa64ad4844a0b2698dee82f95f23&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=44e187abae1d4b248fd145ad9cb97ed2 => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:11.901363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d60338-0b02-4f5a-8a89-848d7baca1aa None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:11.953419 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c95da-aaf2-4a70-a944-e68c89e06137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:11.954617 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c95da-aaf2-4a70-a944-e68c89e06137 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:11.955167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c95da-aaf2-4a70-a944-e68c89e06137 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:11.955433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c95da-aaf2-4a70-a944-e68c89e06137 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:11.956150 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c95da-aaf2-4a70-a944-e68c89e06137 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:11.956448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c95da-aaf2-4a70-a944-e68c89e06137 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:11.957936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0c95da-aaf2-4a70-a944-e68c89e06137 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:11.975225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 275/547] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:11.980883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532703a-2e5f-43f7-831c-049916c1f3a6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:12.099625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:12.113053 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532703a-2e5f-43f7-831c-049916c1f3a6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:12.158720 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab78b9-67f0-4dc5-9fc3-d65dcee613c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:12.159340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab78b9-67f0-4dc5-9fc3-d65dcee613c2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.159614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab78b9-67f0-4dc5-9fc3-d65dcee613c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.159879 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab78b9-67f0-4dc5-9fc3-d65dcee613c2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.175265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab78b9-67f0-4dc5-9fc3-d65dcee613c2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:12.176779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab78b9-67f0-4dc5-9fc3-d65dcee613c2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:12.190450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-11ab78b9-67f0-4dc5-9fc3-d65dcee613c2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:12.190450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:12.190450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:12.190450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:12.190450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:12.190450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:12.190450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:12.190450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:12.190450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:12.190450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:12.190450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:12.190450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:12.190450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:12.190450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:12.190450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:12.190450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:12.191443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:12.193530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:12.193530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:12.193530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:12.193530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:12.193530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:12.193530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:12.193530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:12.193530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:12.193530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:12.193530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:12.193530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 273/548] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:12.197306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c95da-aaf2-4a70-a944-e68c89e06137 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:12.200245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:12.314091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63347ac-e2e3-4952-8ba5-7c5fac490c5b None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.314725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63347ac-e2e3-4952-8ba5-7c5fac490c5b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.315269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63347ac-e2e3-4952-8ba5-7c5fac490c5b None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.319261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 274/549] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:12.325474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d976360-a358-4a63-9872-51df8b994571 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.326199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d976360-a358-4a63-9872-51df8b994571 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.326867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d976360-a358-4a63-9872-51df8b994571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.349443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c95da-aaf2-4a70-a944-e68c89e06137 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:12.362417 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-3d976360-a358-4a63-9872-51df8b994571 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:12.364100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3d976360-a358-4a63-9872-51df8b994571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:12.411277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0458db-1796-4c7e-8c01-bb0823e6c58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:12.411951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0458db-1796-4c7e-8c01-bb0823e6c58f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.412229 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0458db-1796-4c7e-8c01-bb0823e6c58f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.412499 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0458db-1796-4c7e-8c01-bb0823e6c58f None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.413225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0458db-1796-4c7e-8c01-bb0823e6c58f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:12.413525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0458db-1796-4c7e-8c01-bb0823e6c58f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:12.415740 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0458db-1796-4c7e-8c01-bb0823e6c58f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:12.442972 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 276/550] 10.222.0.91 () {56 vars in 1135 bytes} [Thu Sep 8 13:38:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:12.444720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6ff06-3285-49bc-ae87-6850115533e4 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.445318 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6ff06-3285-49bc-ae87-6850115533e4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.446724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6ff06-3285-49bc-ae87-6850115533e4 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.448240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 277/551] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:12.452024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6ff06-3285-49bc-ae87-6850115533e4 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:12.479276 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 275/552] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:12.480760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9912b1e-02a6-4821-9a66-3dbfd862bfe0 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.481060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9912b1e-02a6-4821-9a66-3dbfd862bfe0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.481340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9912b1e-02a6-4821-9a66-3dbfd862bfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.512363 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b9912b1e-02a6-4821-9a66-3dbfd862bfe0 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:12.513573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b9912b1e-02a6-4821-9a66-3dbfd862bfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:12.574807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:12.642703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 276/553] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:12.648726 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6ff06-3285-49bc-ae87-6850115533e4 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:12.694675 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5dd3d-44db-4f47-a530-d9c39e3edf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:12.695317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5dd3d-44db-4f47-a530-d9c39e3edf6b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.695621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5dd3d-44db-4f47-a530-d9c39e3edf6b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.695917 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5dd3d-44db-4f47-a530-d9c39e3edf6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.778122 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5dd3d-44db-4f47-a530-d9c39e3edf6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:12.779615 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5dd3d-44db-4f47-a530-d9c39e3edf6b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:12.821979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9912b1e-02a6-4821-9a66-3dbfd862bfe0 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:12.835541 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:12.861348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:12.936300 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 278/554] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:12.944621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5dd3d-44db-4f47-a530-d9c39e3edf6b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:12.990091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9912b1e-02a6-4821-9a66-3dbfd862bfe0 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.031162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67fd5c-5cf6-452a-b20c-930d53bdc842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.031607 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67fd5c-5cf6-452a-b20c-930d53bdc842 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.031858 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67fd5c-5cf6-452a-b20c-930d53bdc842 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.032165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67fd5c-5cf6-452a-b20c-930d53bdc842 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.047337 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67fd5c-5cf6-452a-b20c-930d53bdc842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:13.053472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-0a67fd5c-5cf6-452a-b20c-930d53bdc842 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:13.053472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:13.053472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:13.053472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:13.053472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:13.053472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:13.053472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:13.053472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:13.053472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:13.053472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:13.053472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:13.053472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:13.053472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:13.053472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:13.053472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:13.053472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:13.054442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:13.054442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:13.054442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:13.054442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:13.054442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:13.054442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:13.054442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:13.054442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:13.054442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:13.054442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:13.054442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:13.054442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:13.054442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:13.054442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:13.054442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:13.054442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:13.060111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:13.060111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:13.060111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:13.060111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:13.060111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:13.060111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:13.060111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:13.060111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:13.060111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:13.060111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:13.060111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:13.060111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:13.060111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 277/555] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:13.060111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67fd5c-5cf6-452a-b20c-930d53bdc842 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.109189 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5dd3d-44db-4f47-a530-d9c39e3edf6b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.160902 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b56b8-8cf0-47ae-a151-155dbe80c8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.161557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b56b8-8cf0-47ae-a151-155dbe80c8e4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.161837 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b56b8-8cf0-47ae-a151-155dbe80c8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.162106 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b56b8-8cf0-47ae-a151-155dbe80c8e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.163797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:13.174857 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b56b8-8cf0-47ae-a151-155dbe80c8e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.176146 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b56b8-8cf0-47ae-a151-155dbe80c8e4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.194154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-474b56b8-8cf0-47ae-a151-155dbe80c8e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:13.194154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:13.194154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:13.194154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:13.194154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:13.194154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:13.194154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:13.194154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:13.194154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:13.194154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:13.194154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:13.194154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:13.194154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:13.194154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:13.194154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:13.194154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:13.195531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:13.196503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:13.196503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:13.196503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:13.200893 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 279/556] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:13.202144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b56b8-8cf0-47ae-a151-155dbe80c8e4 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.220954 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67fd5c-5cf6-452a-b20c-930d53bdc842 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.224879 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:13.264361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4adde5e-a1fc-4f6f-a550-f9c32487d4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.264927 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4adde5e-a1fc-4f6f-a550-f9c32487d4f7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.265201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4adde5e-a1fc-4f6f-a550-f9c32487d4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.265464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4adde5e-a1fc-4f6f-a550-f9c32487d4f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.277921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4adde5e-a1fc-4f6f-a550-f9c32487d4f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:13.284888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-b4adde5e-a1fc-4f6f-a550-f9c32487d4f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:13.284888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:13.284888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:13.284888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:13.284888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:13.284888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:13.284888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:13.284888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:13.284888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:13.284888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:13.284888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:13.284888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:13.284888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:13.284888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:13.284888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:13.284888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:13.286315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:13.286315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:13.286315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:13.286315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:13.286315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:13.286315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:13.286315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:13.286315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:13.286315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:13.286315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:13.286315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:13.286315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:13.286315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:13.286315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:13.286315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:13.286315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:13.287575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:13.287575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:13.287575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:13.287575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:13.287575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:13.287575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:13.287575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:13.287575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:13.287575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:13.287575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:13.287575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:13.287575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:13.291395 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 278/557] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:13.292804 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4adde5e-a1fc-4f6f-a550-f9c32487d4f7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.340529 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b56b8-8cf0-47ae-a151-155dbe80c8e4 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.393838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c348eea-f5f1-43ed-9705-40148f846173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.394476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c348eea-f5f1-43ed-9705-40148f846173 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.394757 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c348eea-f5f1-43ed-9705-40148f846173 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.395076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c348eea-f5f1-43ed-9705-40148f846173 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.395767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c348eea-f5f1-43ed-9705-40148f846173 None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:13.396073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c348eea-f5f1-43ed-9705-40148f846173 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.398347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c348eea-f5f1-43ed-9705-40148f846173 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.411398 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 280/558] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:13.412259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c348eea-f5f1-43ed-9705-40148f846173 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.429063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4adde5e-a1fc-4f6f-a550-f9c32487d4f7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.474261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:13.490288 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db62c93-8604-41fe-acfb-a881a1f47ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.493490 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db62c93-8604-41fe-acfb-a881a1f47ef2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.493490 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db62c93-8604-41fe-acfb-a881a1f47ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.493490 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db62c93-8604-41fe-acfb-a881a1f47ef2 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.493490 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db62c93-8604-41fe-acfb-a881a1f47ef2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:13.493490 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db62c93-8604-41fe-acfb-a881a1f47ef2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.495108 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db62c93-8604-41fe-acfb-a881a1f47ef2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.515588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 279/559] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:13.516999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db62c93-8604-41fe-acfb-a881a1f47ef2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.527755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:13.557849 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c348eea-f5f1-43ed-9705-40148f846173 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.609959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511d7e5-065b-47f5-a0e2-21257256587e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.611196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511d7e5-065b-47f5-a0e2-21257256587e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.611478 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511d7e5-065b-47f5-a0e2-21257256587e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.611746 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511d7e5-065b-47f5-a0e2-21257256587e None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.612569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511d7e5-065b-47f5-a0e2-21257256587e None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:13.612871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511d7e5-065b-47f5-a0e2-21257256587e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.614792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511d7e5-065b-47f5-a0e2-21257256587e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.623822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 281/560] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:13.630064 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75ec35-c371-4e98-bebb-7e4c05e4a172 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.630352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75ec35-c371-4e98-bebb-7e4c05e4a172 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.630630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75ec35-c371-4e98-bebb-7e4c05e4a172 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.632095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 282/561] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:13.632641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75ec35-c371-4e98-bebb-7e4c05e4a172 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.657046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db62c93-8604-41fe-acfb-a881a1f47ef2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.695219 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966142a-9d82-4da4-9c24-327530067b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.695843 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966142a-9d82-4da4-9c24-327530067b7f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.696144 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966142a-9d82-4da4-9c24-327530067b7f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.696416 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966142a-9d82-4da4-9c24-327530067b7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.708372 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0966142a-9d82-4da4-9c24-327530067b7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.709824 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0966142a-9d82-4da4-9c24-327530067b7f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.725706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-0966142a-9d82-4da4-9c24-327530067b7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:13.725706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:13.725706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:13.725706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:13.725706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:13.725706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:13.725706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:13.725706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:13.725706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:13.725706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:13.725706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:13.725706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:13.725706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:13.725706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:13.725706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:13.725706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:13.726727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:13.727797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:13.727797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:13.727797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:13.731344 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 280/562] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:13.732251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966142a-9d82-4da4-9c24-327530067b7f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.759782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75ec35-c371-4e98-bebb-7e4c05e4a172 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.784517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:13.798522 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e7c4b-e9ca-4c35-8932-5477f4f5df66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.803119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e7c4b-e9ca-4c35-8932-5477f4f5df66 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.803119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e7c4b-e9ca-4c35-8932-5477f4f5df66 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.803119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e7c4b-e9ca-4c35-8932-5477f4f5df66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.803119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:13.807982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e7c4b-e9ca-4c35-8932-5477f4f5df66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.809647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e7c4b-e9ca-4c35-8932-5477f4f5df66 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.822458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-953e7c4b-e9ca-4c35-8932-5477f4f5df66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:13.822458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:13.822458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:13.822458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:13.822458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:13.822458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:13.822458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:13.822458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:13.822458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:13.822458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:13.822458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:13.822458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:13.822458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:13.822458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:13.822458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:13.829582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:13.830930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:13.830930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:13.830930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:13.830930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:13.830930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:13.830930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:13.830930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:13.830930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:13.830930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 283/563] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:13.830930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11cf0d-888e-4451-aa94-bde399b7639c None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.830930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11cf0d-888e-4451-aa94-bde399b7639c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.831733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11cf0d-888e-4451-aa94-bde399b7639c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.843650 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-fd11cf0d-888e-4451-aa94-bde399b7639c None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:13.845366 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fd11cf0d-888e-4451-aa94-bde399b7639c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:13.862214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966142a-9d82-4da4-9c24-327530067b7f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.899759 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557d5ca-1f42-4bc7-8865-84283c9c89b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.900531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557d5ca-1f42-4bc7-8865-84283c9c89b5 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.900865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557d5ca-1f42-4bc7-8865-84283c9c89b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.901193 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557d5ca-1f42-4bc7-8865-84283c9c89b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.913065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557d5ca-1f42-4bc7-8865-84283c9c89b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.914536 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557d5ca-1f42-4bc7-8865-84283c9c89b5 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.917377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 284/564] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:13.927124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11cf0d-888e-4451-aa94-bde399b7639c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.940273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-7557d5ca-1f42-4bc7-8865-84283c9c89b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:13.940273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:13.940273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:13.940273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:13.940273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:13.940273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:13.940273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:13.940273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:13.940273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:13.940273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:13.940273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:13.940273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:13.940273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:13.940273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:13.940273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:13.940273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:13.941281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:13.942157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:13.942157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:13.942157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:13.942157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 281/565] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:13.942157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557d5ca-1f42-4bc7-8865-84283c9c89b5 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.058278 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11cf0d-888e-4451-aa94-bde399b7639c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.081777 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:14.090159 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557d5ca-1f42-4bc7-8865-84283c9c89b5 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.102699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:14.114391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f8ae5-459d-4b41-9c3d-e4ffcf4c8ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.115533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f8ae5-459d-4b41-9c3d-e4ffcf4c8ed9 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.116018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f8ae5-459d-4b41-9c3d-e4ffcf4c8ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.116614 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f8ae5-459d-4b41-9c3d-e4ffcf4c8ed9 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.117970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f8ae5-459d-4b41-9c3d-e4ffcf4c8ed9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:14.118533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f8ae5-459d-4b41-9c3d-e4ffcf4c8ed9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:14.121912 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f8ae5-459d-4b41-9c3d-e4ffcf4c8ed9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:14.125457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088857f8-2803-4158-a1a1-c690afedb0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.127234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088857f8-2803-4158-a1a1-c690afedb0ba None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.127234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088857f8-2803-4158-a1a1-c690afedb0ba None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.127234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088857f8-2803-4158-a1a1-c690afedb0ba None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.127929 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088857f8-2803-4158-a1a1-c690afedb0ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:14.128282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088857f8-2803-4158-a1a1-c690afedb0ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:14.129746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088857f8-2803-4158-a1a1-c690afedb0ba None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:14.142506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 285/566] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:14.142506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f8ae5-459d-4b41-9c3d-e4ffcf4c8ed9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.150637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 282/567] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:14.154949 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088857f8-2803-4158-a1a1-c690afedb0ba None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.282046 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f8ae5-459d-4b41-9c3d-e4ffcf4c8ed9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.290262 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088857f8-2803-4158-a1a1-c690afedb0ba None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.323419 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2b585-bafc-499f-b629-1b2154fef6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.324274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2b585-bafc-499f-b629-1b2154fef6a9 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.324800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2b585-bafc-499f-b629-1b2154fef6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.325263 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2b585-bafc-499f-b629-1b2154fef6a9 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.326187 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2b585-bafc-499f-b629-1b2154fef6a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:14.326704 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2b585-bafc-499f-b629-1b2154fef6a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:14.328296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2b585-bafc-499f-b629-1b2154fef6a9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:14.330713 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca23813-6564-413d-a212-98387f9f0f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.331377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca23813-6564-413d-a212-98387f9f0f7b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.331655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca23813-6564-413d-a212-98387f9f0f7b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.331926 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca23813-6564-413d-a212-98387f9f0f7b None admin] PATH_INFO: `/v3/users/glance` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.340206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca23813-6564-413d-a212-98387f9f0f7b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:14.341672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca23813-6564-413d-a212-98387f9f0f7b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:14.343961 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 286/568] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:14.345420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2b585-bafc-499f-b629-1b2154fef6a9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.365920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-0ca23813-6564-413d-a212-98387f9f0f7b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:38:14.365920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:14.365920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:14.365920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:14.365920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:14.365920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:14.365920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:14.365920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:14.365920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:14.365920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:14.365920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:14.365920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:14.365920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:14.365920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:14.365920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:14.365920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:14.367250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:14.368205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:14.368205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:14.368205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:14.368205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:14.368205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:14.368205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:14.368205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:14.368205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:14.368205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:38:14.368205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:14.368205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 283/569] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:14.368205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca23813-6564-413d-a212-98387f9f0f7b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.389831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:14.419402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:14.495854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca23813-6564-413d-a212-98387f9f0f7b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.497156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2b585-bafc-499f-b629-1b2154fef6a9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.532121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd97de-e1f8-43ea-92d2-cf417b79540c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.532746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd97de-e1f8-43ea-92d2-cf417b79540c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.533020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd97de-e1f8-43ea-92d2-cf417b79540c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.533291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd97de-e1f8-43ea-92d2-cf417b79540c None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.536727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ef86c-4f0f-4c5a-a61e-5a802d8f99c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.537330 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ef86c-4f0f-4c5a-a61e-5a802d8f99c9 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.537607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ef86c-4f0f-4c5a-a61e-5a802d8f99c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.537877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ef86c-4f0f-4c5a-a61e-5a802d8f99c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.538599 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ef86c-4f0f-4c5a-a61e-5a802d8f99c9 None admin] RBAC: Adding query filter params (role.id=44e187abae1d4b248fd145ad9cb97ed2, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=b3b5ae758fe847adaae0edaa0be095c9) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:14.538920 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ef86c-4f0f-4c5a-a61e-5a802d8f99c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:14.540376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ef86c-4f0f-4c5a-a61e-5a802d8f99c9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:14.545347 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bd97de-e1f8-43ea-92d2-cf417b79540c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:14.546282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 287/570] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:14 2022] GET /identity/v3/role_assignments?user.id=b3b5ae758fe847adaae0edaa0be095c9&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=44e187abae1d4b248fd145ad9cb97ed2 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:14.547142 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ef86c-4f0f-4c5a-a61e-5a802d8f99c9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.554774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-07bd97de-e1f8-43ea-92d2-cf417b79540c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:14.554774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:14.554774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:14.554774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:14.554774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:14.554774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:14.554774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:14.554774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:14.554774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:14.554774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:14.554774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:14.554774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:14.554774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:14.554774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:14.554774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:14.554774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:14.555814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:14.555814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:14.555814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:14.555814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:14.555814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:14.555814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:14.555814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:14.555814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:14.555814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:14.555814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:14.555814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:14.555814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:14.555814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:14.555814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:14.555814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:14.555814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:14.559214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:14.559214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:14.559214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:14.559214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:14.559214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:14.559214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:14.559214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:14.559214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:14.559214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:14.559214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:14.559214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:14.559214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:14.559214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 284/571] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:14.559214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd97de-e1f8-43ea-92d2-cf417b79540c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.669280 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:14.701557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ef86c-4f0f-4c5a-a61e-5a802d8f99c9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.702603 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bd97de-e1f8-43ea-92d2-cf417b79540c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.715822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:14.748435 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8ebc6e-6010-44d1-ae84-39276193cb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.748435 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ebc6e-6010-44d1-ae84-39276193cb6d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.748896 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ebc6e-6010-44d1-ae84-39276193cb6d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.749131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8ebc6e-6010-44d1-ae84-39276193cb6d None admin] PATH_INFO: `/v3/users/nova` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.749255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf5461-9aca-40b3-a3a1-bd08e931d2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.749825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf5461-9aca-40b3-a3a1-bd08e931d2a2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.750106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf5461-9aca-40b3-a3a1-bd08e931d2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.750382 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf5461-9aca-40b3-a3a1-bd08e931d2a2 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.751194 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf5461-9aca-40b3-a3a1-bd08e931d2a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:14.751500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf5461-9aca-40b3-a3a1-bd08e931d2a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:14.752982 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf5461-9aca-40b3-a3a1-bd08e931d2a2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:14.759148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ebc6e-6010-44d1-ae84-39276193cb6d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:14.760633 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8ebc6e-6010-44d1-ae84-39276193cb6d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:14.772824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-2e8ebc6e-6010-44d1-ae84-39276193cb6d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:14.772824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:14.772824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:14.772824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:14.772824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:14.772824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:14.772824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:14.772824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:14.772824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:14.772824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:14.772824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:14.772824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:14.772824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:14.772824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:14.772824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:14.772824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:14.773858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:14.775510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:14.775510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:14.775510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:14.775510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:14.775510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:14.775510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:14.775510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:14.775510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:14.775510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:14.775510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:14.775510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 288/572] 10.222.0.91 () {56 vars in 1093 bytes} [Thu Sep 8 13:38:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:14.776095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3985a5-0dbf-4cdd-bcab-05d73cf44d35 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.776095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3985a5-0dbf-4cdd-bcab-05d73cf44d35 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.776240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3985a5-0dbf-4cdd-bcab-05d73cf44d35 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.777615 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 289/573] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:14.778085 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3985a5-0dbf-4cdd-bcab-05d73cf44d35 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.784420 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 285/574] 10.222.0.91 () {56 vars in 1137 bytes} [Thu Sep 8 13:38:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:14.784420 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf5461-9aca-40b3-a3a1-bd08e931d2a2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.894948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3985a5-0dbf-4cdd-bcab-05d73cf44d35 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.895621 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf5461-9aca-40b3-a3a1-bd08e931d2a2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.928419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:14.931108 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80786ddb-e68b-4249-9470-05e2fa14e3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.931889 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80786ddb-e68b-4249-9470-05e2fa14e3c4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.932166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80786ddb-e68b-4249-9470-05e2fa14e3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.932433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80786ddb-e68b-4249-9470-05e2fa14e3c4 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.933100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80786ddb-e68b-4249-9470-05e2fa14e3c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:14.933401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80786ddb-e68b-4249-9470-05e2fa14e3c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:14.934856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80786ddb-e68b-4249-9470-05e2fa14e3c4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:14.939319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d889d-7381-45b2-93d2-6096c9815aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.939319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d889d-7381-45b2-93d2-6096c9815aea None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.939319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d889d-7381-45b2-93d2-6096c9815aea None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.939319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d889d-7381-45b2-93d2-6096c9815aea None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.939645 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d889d-7381-45b2-93d2-6096c9815aea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:14.939645 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d889d-7381-45b2-93d2-6096c9815aea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:14.939645 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d889d-7381-45b2-93d2-6096c9815aea None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:14.948861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 290/575] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:14.948861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc83e9-2e79-40e2-bff6-fa940b87cc7e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.948861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc83e9-2e79-40e2-bff6-fa940b87cc7e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.948861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc83e9-2e79-40e2-bff6-fa940b87cc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.967130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 286/576] 10.222.0.91 () {56 vars in 1133 bytes} [Thu Sep 8 13:38:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:14.967130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d889d-7381-45b2-93d2-6096c9815aea None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.967407 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-56bc83e9-2e79-40e2-bff6-fa940b87cc7e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:14.968672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-56bc83e9-2e79-40e2-bff6-fa940b87cc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:15.005415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:15.059215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 291/577] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:15.060848 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc83e9-2e79-40e2-bff6-fa940b87cc7e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:15.095633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d889d-7381-45b2-93d2-6096c9815aea None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:15.140698 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc37d03-1601-416f-a07a-53132e69553f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:15.141359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc37d03-1601-416f-a07a-53132e69553f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:15.142538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc37d03-1601-416f-a07a-53132e69553f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:15.142814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc37d03-1601-416f-a07a-53132e69553f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:15.156862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc37d03-1601-416f-a07a-53132e69553f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:15.159060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc37d03-1601-416f-a07a-53132e69553f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:15.180403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-ffc37d03-1601-416f-a07a-53132e69553f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:15.180403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:15.180403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:15.180403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:15.180403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:15.180403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:15.180403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:15.180403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:15.180403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:15.180403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:15.180403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:15.180403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:15.180403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:15.180403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:15.180403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:15.180403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:15.182036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:15.183455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:15.183455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:15.183455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:15.186968 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 287/578] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:15.187954 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc37d03-1601-416f-a07a-53132e69553f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:15.192572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc83e9-2e79-40e2-bff6-fa940b87cc7e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:15.231482 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4331393-4d52-4467-8bde-e497dc8a6cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:15.232490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4331393-4d52-4467-8bde-e497dc8a6cc0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:15.232973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4331393-4d52-4467-8bde-e497dc8a6cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:15.233533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4331393-4d52-4467-8bde-e497dc8a6cc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:15.245060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4331393-4d52-4467-8bde-e497dc8a6cc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:15.246998 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4331393-4d52-4467-8bde-e497dc8a6cc0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:15.266022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-d4331393-4d52-4467-8bde-e497dc8a6cc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:15.266022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:15.266022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:15.266022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:15.266022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:15.266022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:15.266022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:15.266022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:15.266022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:15.266022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:15.266022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:15.266022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:15.266022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:15.266022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:15.266022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:15.266022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:15.267761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:15.269218 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:15.269218 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:15.269218 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:15.272129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 292/579] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:15.273589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4331393-4d52-4467-8bde-e497dc8a6cc0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:15.300573 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:15.303211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:15.324113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc37d03-1601-416f-a07a-53132e69553f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:15.381439 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2212180-f213-4818-860c-b8aa99cb2d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:15.382090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2212180-f213-4818-860c-b8aa99cb2d14 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:15.382364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2212180-f213-4818-860c-b8aa99cb2d14 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:15.382635 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2212180-f213-4818-860c-b8aa99cb2d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:15.396973 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2212180-f213-4818-860c-b8aa99cb2d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:15.398424 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2212180-f213-4818-860c-b8aa99cb2d14 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:15.420652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-f2212180-f213-4818-860c-b8aa99cb2d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:15.420652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:15.420652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:15.420652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:15.420652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:15.420652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:15.420652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:15.420652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:15.420652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:15.420652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:15.420652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:15.420652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:15.420652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:15.420652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:15.420652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:15.420652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:15.421773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:15.424950 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:15.424950 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:15.424950 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:15.424950 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 288/580] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:15.424950 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2212180-f213-4818-860c-b8aa99cb2d14 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:15.429938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4331393-4d52-4467-8bde-e497dc8a6cc0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:15.468181 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724c004-e6bf-4ed1-91bf-6f2b0eef4655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:15.468680 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724c004-e6bf-4ed1-91bf-6f2b0eef4655 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:15.468953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724c004-e6bf-4ed1-91bf-6f2b0eef4655 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:15.469249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724c004-e6bf-4ed1-91bf-6f2b0eef4655 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:15.470384 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724c004-e6bf-4ed1-91bf-6f2b0eef4655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:15.470684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724c004-e6bf-4ed1-91bf-6f2b0eef4655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:15.472107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724c004-e6bf-4ed1-91bf-6f2b0eef4655 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:15.489524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 293/581] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:15.490416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724c004-e6bf-4ed1-91bf-6f2b0eef4655 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:15.558249 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:15.569733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2212180-f213-4818-860c-b8aa99cb2d14 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:15.595737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:15.623725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faedced-beec-4e25-9ed5-99f176dd137c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:15.624532 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faedced-beec-4e25-9ed5-99f176dd137c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:15.624991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faedced-beec-4e25-9ed5-99f176dd137c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:15.625406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faedced-beec-4e25-9ed5-99f176dd137c None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:15.626837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faedced-beec-4e25-9ed5-99f176dd137c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:15.627303 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faedced-beec-4e25-9ed5-99f176dd137c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:15.629263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faedced-beec-4e25-9ed5-99f176dd137c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:15.636564 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724c004-e6bf-4ed1-91bf-6f2b0eef4655 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:15.648943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 289/582] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:15.648943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faedced-beec-4e25-9ed5-99f176dd137c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:15.685734 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280986c-0b65-478f-be22-feacfbc46959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:15.686382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280986c-0b65-478f-be22-feacfbc46959 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:15.686704 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280986c-0b65-478f-be22-feacfbc46959 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:15.687006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280986c-0b65-478f-be22-feacfbc46959 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:15.699949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c280986c-0b65-478f-be22-feacfbc46959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:15.708148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-c280986c-0b65-478f-be22-feacfbc46959 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:15.708148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:15.708148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:15.708148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:15.708148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:15.708148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:15.708148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:15.708148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:15.708148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:15.708148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:15.708148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:15.708148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:15.708148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:15.708148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:15.708148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:15.708148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:15.709319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:15.709319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:15.709319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:15.709319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:15.709319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:15.709319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:15.709319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:15.709319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:15.709319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:15.709319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:15.709319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:15.709319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:15.709319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:15.709319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:15.709319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:15.709319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:15.710281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:15.710281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:15.710281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:15.710281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:15.710281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:15.710281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:15.710281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:15.710281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:15.710281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:15.710281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:15.710281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:15.710281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:15.717758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 294/583] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:15.717758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280986c-0b65-478f-be22-feacfbc46959 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:15.789158 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faedced-beec-4e25-9ed5-99f176dd137c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:15.839773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716cde3-e496-4019-89be-1551d1324e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:15.840378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716cde3-e496-4019-89be-1551d1324e67 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:15.840647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716cde3-e496-4019-89be-1551d1324e67 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:15.840912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716cde3-e496-4019-89be-1551d1324e67 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:15.841550 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716cde3-e496-4019-89be-1551d1324e67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:15.841842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716cde3-e496-4019-89be-1551d1324e67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:15.843230 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716cde3-e496-4019-89be-1551d1324e67 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:15.847204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280986c-0b65-478f-be22-feacfbc46959 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:15.860339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 290/584] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:15.860339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716cde3-e496-4019-89be-1551d1324e67 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:15.875970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:15.887504 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ab54b-634c-4f0e-80f8-267fb4703fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:15.888367 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ab54b-634c-4f0e-80f8-267fb4703fa2 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:15.888646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ab54b-634c-4f0e-80f8-267fb4703fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:15.888917 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ab54b-634c-4f0e-80f8-267fb4703fa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:15.890874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:15.896518 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ab54b-634c-4f0e-80f8-267fb4703fa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:15.898035 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ab54b-634c-4f0e-80f8-267fb4703fa2 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:15.909974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-ce5ab54b-634c-4f0e-80f8-267fb4703fa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:15.909974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:15.909974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:15.909974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:15.909974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:15.909974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:15.909974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:15.909974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:15.909974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:15.909974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:15.909974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:15.909974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:15.909974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:15.909974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:15.909974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:15.911093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:15.912048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:15.912048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:15.912048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:15.912048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:15.912048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:15.912048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:15.912048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:15.912048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:15.912048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 295/585] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:15.912671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ab54b-634c-4f0e-80f8-267fb4703fa2 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:15.985987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716cde3-e496-4019-89be-1551d1324e67 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:16.027025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbca24f-6824-4f4d-b069-763df0e15bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:16.027796 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbca24f-6824-4f4d-b069-763df0e15bdc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.028075 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbca24f-6824-4f4d-b069-763df0e15bdc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.028345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbca24f-6824-4f4d-b069-763df0e15bdc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.036798 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbca24f-6824-4f4d-b069-763df0e15bdc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:16.037387 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ab54b-634c-4f0e-80f8-267fb4703fa2 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:16.038459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbca24f-6824-4f4d-b069-763df0e15bdc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:16.055958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-7fbca24f-6824-4f4d-b069-763df0e15bdc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:38:16.055958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:16.055958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:16.055958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:16.055958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:16.055958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:16.055958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:16.055958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:16.055958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:16.055958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:16.055958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:16.055958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:16.055958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:16.055958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:16.055958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:16.055958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:16.057569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:16.059240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:16.059240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:16.059240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:16.059240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:16.059240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:16.059240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:16.059240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:16.059240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:16.059240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:38:16.059240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:16.059240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 291/586] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:16.059240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbca24f-6824-4f4d-b069-763df0e15bdc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:16.083992 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59557741-0e0c-45d5-8849-2e6befd3d9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:16.084736 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59557741-0e0c-45d5-8849-2e6befd3d9ef None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.084859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59557741-0e0c-45d5-8849-2e6befd3d9ef None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.085168 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59557741-0e0c-45d5-8849-2e6befd3d9ef None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.085790 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59557741-0e0c-45d5-8849-2e6befd3d9ef None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:16.086170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59557741-0e0c-45d5-8849-2e6befd3d9ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:16.087620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59557741-0e0c-45d5-8849-2e6befd3d9ef None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:16.097526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 296/587] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:16.098460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59557741-0e0c-45d5-8849-2e6befd3d9ef None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:16.157897 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:16.163655 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:16.186748 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbca24f-6824-4f4d-b069-763df0e15bdc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:16.216183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59557741-0e0c-45d5-8849-2e6befd3d9ef None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:16.227438 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d8cf4-dd85-4ba4-ad07-ceeaee7ddbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:16.228384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d8cf4-dd85-4ba4-ad07-ceeaee7ddbd8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.228676 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d8cf4-dd85-4ba4-ad07-ceeaee7ddbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.228953 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d8cf4-dd85-4ba4-ad07-ceeaee7ddbd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.236016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d8cf4-dd85-4ba4-ad07-ceeaee7ddbd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:16.237480 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1d8cf4-dd85-4ba4-ad07-ceeaee7ddbd8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:16.249015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-3e1d8cf4-dd85-4ba4-ad07-ceeaee7ddbd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:16.249015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:16.249015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:16.249015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:16.249015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:16.249015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:16.249015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:16.249015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:16.249015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:16.249015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:16.249015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:16.249015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:16.249015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:16.249015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:16.249015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:16.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:16.251827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:16.251827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:16.251827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:16.251827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:16.251827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:16.251827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:16.251827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:16.251827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:16.255541 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 292/588] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:16.256052 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d8cf4-dd85-4ba4-ad07-ceeaee7ddbd8 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:16.259832 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b4e4c-1600-4434-a2aa-85cc58ecc542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:16.260547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b4e4c-1600-4434-a2aa-85cc58ecc542 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.260921 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b4e4c-1600-4434-a2aa-85cc58ecc542 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.261288 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b4e4c-1600-4434-a2aa-85cc58ecc542 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.262093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b4e4c-1600-4434-a2aa-85cc58ecc542 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:16.262490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b4e4c-1600-4434-a2aa-85cc58ecc542 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:16.264187 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b4e4c-1600-4434-a2aa-85cc58ecc542 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:16.283903 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 297/589] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:16.283903 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b4e4c-1600-4434-a2aa-85cc58ecc542 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:16.366528 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1d8cf4-dd85-4ba4-ad07-ceeaee7ddbd8 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:16.401841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc72bd-4c03-4d25-b3ce-35ca0590ca18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:16.402450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc72bd-4c03-4d25-b3ce-35ca0590ca18 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.402730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc72bd-4c03-4d25-b3ce-35ca0590ca18 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.403102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc72bd-4c03-4d25-b3ce-35ca0590ca18 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.403829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc72bd-4c03-4d25-b3ce-35ca0590ca18 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:16.404131 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc72bd-4c03-4d25-b3ce-35ca0590ca18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:16.405637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b4e4c-1600-4434-a2aa-85cc58ecc542 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:16.405758 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc72bd-4c03-4d25-b3ce-35ca0590ca18 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:16.407977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:16.425099 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 293/590] 10.222.0.91 () {56 vars in 1137 bytes} [Thu Sep 8 13:38:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:16.426553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:16.427740 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc72bd-4c03-4d25-b3ce-35ca0590ca18 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:16.446018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed06c2-916f-4031-a670-e999542215f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:16.446654 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed06c2-916f-4031-a670-e999542215f8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.446962 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed06c2-916f-4031-a670-e999542215f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.447233 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed06c2-916f-4031-a670-e999542215f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.458449 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed06c2-916f-4031-a670-e999542215f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:16.460028 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed06c2-916f-4031-a670-e999542215f8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:16.477678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-5fed06c2-916f-4031-a670-e999542215f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:16.477678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:16.477678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:16.477678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:16.477678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:16.477678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:16.477678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:16.477678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:16.477678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:16.477678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:16.477678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:16.477678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:16.477678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:16.477678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:16.477678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:16.477678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:16.478756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:16.479777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:16.479777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:16.479777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:16.479777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 298/591] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:16.479777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed06c2-916f-4031-a670-e999542215f8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:16.559340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc72bd-4c03-4d25-b3ce-35ca0590ca18 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:16.599965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089612c-d4d0-405a-86f4-2d7a6e26b729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:16.600706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089612c-d4d0-405a-86f4-2d7a6e26b729 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.601099 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089612c-d4d0-405a-86f4-2d7a6e26b729 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.601467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089612c-d4d0-405a-86f4-2d7a6e26b729 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.601705 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed06c2-916f-4031-a670-e999542215f8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:16.602330 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089612c-d4d0-405a-86f4-2d7a6e26b729 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:16.602733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089612c-d4d0-405a-86f4-2d7a6e26b729 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:16.604376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089612c-d4d0-405a-86f4-2d7a6e26b729 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:16.621403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 294/592] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:16.621403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089612c-d4d0-405a-86f4-2d7a6e26b729 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:16.646219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e0400-47ef-41fe-9262-03980747eb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:16.646928 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e0400-47ef-41fe-9262-03980747eb28 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.647219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e0400-47ef-41fe-9262-03980747eb28 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.647488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e0400-47ef-41fe-9262-03980747eb28 None admin] PATH_INFO: `/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/01b060c70e1b440186eeac37d9c34567/roles/44e187abae1d4b248fd145ad9cb97ed2` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.663680 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e0400-47ef-41fe-9262-03980747eb28 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb2bd53d40841eeb1653bd5927fb189, user_id=01b060c70e1b440186eeac37d9c34567, role_id=44e187abae1d4b248fd145ad9cb97ed2)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:16.665138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e0400-47ef-41fe-9262-03980747eb28 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:16.688269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:16.737651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089612c-d4d0-405a-86f4-2d7a6e26b729 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:16.770576 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01f5f1-d779-4dc2-9e99-00daa013fef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:16.771227 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01f5f1-d779-4dc2-9e99-00daa013fef1 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.771501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01f5f1-d779-4dc2-9e99-00daa013fef1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.771769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01f5f1-d779-4dc2-9e99-00daa013fef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.782689 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01f5f1-d779-4dc2-9e99-00daa013fef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:16.785294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01f5f1-d779-4dc2-9e99-00daa013fef1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:16.797185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 299/593] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:16 2022] PUT /identity/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/01b060c70e1b440186eeac37d9c34567/roles/44e187abae1d4b248fd145ad9cb97ed2 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:16.805964 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e0400-47ef-41fe-9262-03980747eb28 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:16.806130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-0f01f5f1-d779-4dc2-9e99-00daa013fef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:16.806130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:16.806130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:16.806130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:16.806130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:16.806130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:16.806130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:16.806130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:16.806130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:16.806130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:16.806130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:16.806130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:16.806130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:16.806130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:16.806130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:16.806130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:16.807118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:16.808022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:16.808022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:16.808022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:16.808022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 295/594] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:16.808022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01f5f1-d779-4dc2-9e99-00daa013fef1 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:16.824658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:16.922774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e0400-47ef-41fe-9262-03980747eb28 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:16.934018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01f5f1-d779-4dc2-9e99-00daa013fef1 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:16.936804 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:16.964606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e9a5d-fddd-4ef3-bea3-1d625e88a790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:16.965226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e9a5d-fddd-4ef3-bea3-1d625e88a790 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.965504 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e9a5d-fddd-4ef3-bea3-1d625e88a790 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.965777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e9a5d-fddd-4ef3-bea3-1d625e88a790 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.966543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e9a5d-fddd-4ef3-bea3-1d625e88a790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:16.966982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e9a5d-fddd-4ef3-bea3-1d625e88a790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:16.968408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e9a5d-fddd-4ef3-bea3-1d625e88a790 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:16.972498 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f18f8-df87-40c5-bd5d-2a0751d33767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:16.973169 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f18f8-df87-40c5-bd5d-2a0751d33767 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.974853 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f18f8-df87-40c5-bd5d-2a0751d33767 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.974853 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f18f8-df87-40c5-bd5d-2a0751d33767 None admin] PATH_INFO: `/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/38127f41852d412ea6ad270f1618fce2/roles/44e187abae1d4b248fd145ad9cb97ed2` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.984137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 300/595] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:16.985537 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6610f6-12a8-40c8-b33f-95147a1749b8 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.986311 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6610f6-12a8-40c8-b33f-95147a1749b8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.986594 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6610f6-12a8-40c8-b33f-95147a1749b8 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.987888 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 301/596] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:16.988689 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6610f6-12a8-40c8-b33f-95147a1749b8 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:17.000863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f18f8-df87-40c5-bd5d-2a0751d33767 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb2bd53d40841eeb1653bd5927fb189, user_id=38127f41852d412ea6ad270f1618fce2, role_id=44e187abae1d4b248fd145ad9cb97ed2)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:17.003319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f18f8-df87-40c5-bd5d-2a0751d33767 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:17.075076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:17.083481 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 296/597] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:16 2022] PUT /identity/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/38127f41852d412ea6ad270f1618fce2/roles/44e187abae1d4b248fd145ad9cb97ed2 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:17.084545 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f18f8-df87-40c5-bd5d-2a0751d33767 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:17.113526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6610f6-12a8-40c8-b33f-95147a1749b8 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:17.155977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d8985-a5a9-4d5f-b0e1-ff705ad80a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:17.156849 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d8985-a5a9-4d5f-b0e1-ff705ad80a02 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.157273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d8985-a5a9-4d5f-b0e1-ff705ad80a02 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.157769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d8985-a5a9-4d5f-b0e1-ff705ad80a02 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.158702 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d8985-a5a9-4d5f-b0e1-ff705ad80a02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:17.159225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d8985-a5a9-4d5f-b0e1-ff705ad80a02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:17.160843 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d8985-a5a9-4d5f-b0e1-ff705ad80a02 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:17.175709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 302/598] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:17.177329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27311b57-72ac-4ec0-b00a-a612274a0d9f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.177782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27311b57-72ac-4ec0-b00a-a612274a0d9f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.178201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27311b57-72ac-4ec0-b00a-a612274a0d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.202298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-27311b57-72ac-4ec0-b00a-a612274a0d9f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:17.203965 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-27311b57-72ac-4ec0-b00a-a612274a0d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:17.221116 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f18f8-df87-40c5-bd5d-2a0751d33767 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:17.273558 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825eac5-a51f-47f4-8b0c-ed4715a43f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:17.274702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825eac5-a51f-47f4-8b0c-ed4715a43f79 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.275077 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825eac5-a51f-47f4-8b0c-ed4715a43f79 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.275637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825eac5-a51f-47f4-8b0c-ed4715a43f79 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.285644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 303/599] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:17.286038 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825eac5-a51f-47f4-8b0c-ed4715a43f79 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:17.286369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27311b57-72ac-4ec0-b00a-a612274a0d9f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:17.287550 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825eac5-a51f-47f4-8b0c-ed4715a43f79 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:17.302996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-e825eac5-a51f-47f4-8b0c-ed4715a43f79 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:38:17.302996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:17.302996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:17.302996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:17.302996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:17.302996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:17.302996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:17.302996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:17.302996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:17.302996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:17.302996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:17.302996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:17.302996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:17.302996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:17.302996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:17.302996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:17.304111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:17.305470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:17.305470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:17.305470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:17.305470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:17.305470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:17.305470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:17.305470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:17.305470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:17.305470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:38:17.305470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:17.305470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 297/600] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:17.311981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825eac5-a51f-47f4-8b0c-ed4715a43f79 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:17.330527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:17.389999 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:17.436067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825eac5-a51f-47f4-8b0c-ed4715a43f79 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:17.451248 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27311b57-72ac-4ec0-b00a-a612274a0d9f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:17.503719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5d710-b9f3-4849-a556-45cf805fb0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:17.504802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5d710-b9f3-4849-a556-45cf805fb0f1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.506728 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5d710-b9f3-4849-a556-45cf805fb0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.508041 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5d710-b9f3-4849-a556-45cf805fb0f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.521551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5d710-b9f3-4849-a556-45cf805fb0f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:17.524274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5d710-b9f3-4849-a556-45cf805fb0f1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:17.546862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-94c5d710-b9f3-4849-a556-45cf805fb0f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:17.546862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:17.546862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:17.546862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:17.546862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:17.546862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:17.546862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:17.546862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:17.546862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:17.546862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:17.546862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:17.546862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:17.546862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:17.546862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:17.546862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:17.548523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:17.549632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:17.549632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:17.549632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:17.549632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:17.549632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:17.549632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:17.549632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:17.549632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:17.551784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 304/601] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:17.556301 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33dacb-6977-4e91-afd6-60a2ccd30e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:17.556301 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33dacb-6977-4e91-afd6-60a2ccd30e1a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.556301 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33dacb-6977-4e91-afd6-60a2ccd30e1a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.556301 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33dacb-6977-4e91-afd6-60a2ccd30e1a None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.556702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33dacb-6977-4e91-afd6-60a2ccd30e1a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:17.556702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33dacb-6977-4e91-afd6-60a2ccd30e1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:17.556702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33dacb-6977-4e91-afd6-60a2ccd30e1a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:17.557894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5d710-b9f3-4849-a556-45cf805fb0f1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:17.598292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 298/602] 10.222.0.91 () {56 vars in 1139 bytes} [Thu Sep 8 13:38:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:17.606615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33dacb-6977-4e91-afd6-60a2ccd30e1a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:17.704416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5d710-b9f3-4849-a556-45cf805fb0f1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:17.710468 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:17.735205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:17.745093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b84a1-8fd6-44cd-8c0e-35cfe845f45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:17.746203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b84a1-8fd6-44cd-8c0e-35cfe845f45a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.746677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b84a1-8fd6-44cd-8c0e-35cfe845f45a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.747263 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b84a1-8fd6-44cd-8c0e-35cfe845f45a None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.748821 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b84a1-8fd6-44cd-8c0e-35cfe845f45a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:17.749303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b84a1-8fd6-44cd-8c0e-35cfe845f45a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:17.750934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b84a1-8fd6-44cd-8c0e-35cfe845f45a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:17.751898 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33dacb-6977-4e91-afd6-60a2ccd30e1a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:17.770929 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 305/603] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:17.770929 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b84a1-8fd6-44cd-8c0e-35cfe845f45a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:17.802735 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d42c1-90cc-4835-84db-05ace6be034b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:17.803759 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d42c1-90cc-4835-84db-05ace6be034b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.804506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d42c1-90cc-4835-84db-05ace6be034b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.804860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d42c1-90cc-4835-84db-05ace6be034b None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.817526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d42c1-90cc-4835-84db-05ace6be034b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:17.822549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-9b5d42c1-90cc-4835-84db-05ace6be034b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:17.822549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:17.822549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:17.822549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:17.822549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:17.822549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:17.822549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:17.822549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:17.822549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:17.822549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:17.822549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:17.822549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:17.822549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:17.822549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:17.822549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:17.822549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:17.827206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:17.827206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:17.827206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:17.827206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:17.827206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:17.827206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:17.827206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:17.827206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:17.827206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:17.827206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:17.827206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:17.827206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:17.827206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:17.827206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:17.827206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:17.827206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:17.828268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:17.828268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:17.828268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:17.828268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:17.828268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:17.828268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:17.828268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:17.828268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:17.828268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:17.828268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:17.828268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:17.828268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:17.829319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 299/604] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:17.830482 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d42c1-90cc-4835-84db-05ace6be034b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:17.906879 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b84a1-8fd6-44cd-8c0e-35cfe845f45a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:17.950132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8021a-c5e1-4e58-9954-79c8112631e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:17.951405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8021a-c5e1-4e58-9954-79c8112631e5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.952112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8021a-c5e1-4e58-9954-79c8112631e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.952557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8021a-c5e1-4e58-9954-79c8112631e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.958552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d42c1-90cc-4835-84db-05ace6be034b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:17.968449 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8021a-c5e1-4e58-9954-79c8112631e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:17.971167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8021a-c5e1-4e58-9954-79c8112631e5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:17.991343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-ddb8021a-c5e1-4e58-9954-79c8112631e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:17.991343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:17.991343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:17.991343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:17.991343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:17.991343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:17.991343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:17.991343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:17.991343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:17.991343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:17.991343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:17.991343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:17.991343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:17.991343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:17.991343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:17.991343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:17.992569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:17.993647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:17.993647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:17.993647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:17.995050 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:17.995396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 306/605] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:17.997130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8021a-c5e1-4e58-9954-79c8112631e5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:18.002998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65847840-a7fb-403d-8450-f00c41df5a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:18.003727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65847840-a7fb-403d-8450-f00c41df5a0f None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:18.004267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65847840-a7fb-403d-8450-f00c41df5a0f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:18.004651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65847840-a7fb-403d-8450-f00c41df5a0f None admin] PATH_INFO: `/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/ca225b99584e40f4a4065bf10264974d/roles/44e187abae1d4b248fd145ad9cb97ed2` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:18.026246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65847840-a7fb-403d-8450-f00c41df5a0f None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb2bd53d40841eeb1653bd5927fb189, user_id=ca225b99584e40f4a4065bf10264974d, role_id=44e187abae1d4b248fd145ad9cb97ed2)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:18.028174 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65847840-a7fb-403d-8450-f00c41df5a0f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:18.050812 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:18.113002 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8021a-c5e1-4e58-9954-79c8112631e5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:18.148097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 300/606] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:17 2022] PUT /identity/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/ca225b99584e40f4a4065bf10264974d/roles/44e187abae1d4b248fd145ad9cb97ed2 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:18.150075 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65847840-a7fb-403d-8450-f00c41df5a0f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:18.179019 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1610974-c931-49fb-9798-76bf322243d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:18.182249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1610974-c931-49fb-9798-76bf322243d9 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:18.182923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1610974-c931-49fb-9798-76bf322243d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:18.183830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1610974-c931-49fb-9798-76bf322243d9 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:18.185802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1610974-c931-49fb-9798-76bf322243d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:18.188117 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1610974-c931-49fb-9798-76bf322243d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:18.189867 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1610974-c931-49fb-9798-76bf322243d9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:18.210506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 307/607] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:18.220628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1610974-c931-49fb-9798-76bf322243d9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:18.325724 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65847840-a7fb-403d-8450-f00c41df5a0f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:18.389677 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e357c-4487-438a-b060-1d3e317250ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:18.391391 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e357c-4487-438a-b060-1d3e317250ed None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:18.392011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e357c-4487-438a-b060-1d3e317250ed None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:18.392612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e357c-4487-438a-b060-1d3e317250ed None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:18.393617 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e357c-4487-438a-b060-1d3e317250ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:18.394159 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e357c-4487-438a-b060-1d3e317250ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:18.395906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e357c-4487-438a-b060-1d3e317250ed None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:18.412279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 301/608] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:18.417739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e357c-4487-438a-b060-1d3e317250ed None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:18.437366 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1610974-c931-49fb-9798-76bf322243d9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:18.439307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:18.474035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:18.493802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e914f42-08a4-4607-b5aa-855fcd11a229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:18.494381 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e914f42-08a4-4607-b5aa-855fcd11a229 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:18.494492 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e914f42-08a4-4607-b5aa-855fcd11a229 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:18.494771 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e914f42-08a4-4607-b5aa-855fcd11a229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:18.517216 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e914f42-08a4-4607-b5aa-855fcd11a229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:18.517216 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e914f42-08a4-4607-b5aa-855fcd11a229 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:18.549317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-9e914f42-08a4-4607-b5aa-855fcd11a229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:18.549317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:18.549317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:18.549317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:18.549317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:18.549317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:18.549317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:18.549317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:18.549317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:18.549317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:18.549317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:18.549317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:18.549317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:18.549317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:18.549317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:18.549317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:18.550305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:18.558095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:18.558095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:18.558095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:18.558095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 308/609] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:18.559359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e914f42-08a4-4607-b5aa-855fcd11a229 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:18.589368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e357c-4487-438a-b060-1d3e317250ed None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:18.651123 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b2ab2-6920-42e9-856d-3ad2940ea5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:18.652302 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b2ab2-6920-42e9-856d-3ad2940ea5b6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:18.652659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b2ab2-6920-42e9-856d-3ad2940ea5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:18.653007 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b2ab2-6920-42e9-856d-3ad2940ea5b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:18.661594 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b2ab2-6920-42e9-856d-3ad2940ea5b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:18.664089 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b2ab2-6920-42e9-856d-3ad2940ea5b6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:18.678614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-ed8b2ab2-6920-42e9-856d-3ad2940ea5b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:18.678614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:18.678614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:18.678614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:18.678614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:18.678614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:18.678614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:18.678614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:18.678614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:18.678614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:18.678614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:18.678614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:18.678614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:18.678614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:18.678614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:18.679760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:18.680836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:18.680836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:18.680836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:18.680836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:18.680836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:18.680836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:18.680836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:18.680836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:18.682116 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 302/610] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:18.687515 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b2ab2-6920-42e9-856d-3ad2940ea5b6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:18.769029 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e914f42-08a4-4607-b5aa-855fcd11a229 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:18.806641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa14ee-924c-4606-a63a-0eccceebf8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:18.807449 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa14ee-924c-4606-a63a-0eccceebf8bc None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:18.807830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa14ee-924c-4606-a63a-0eccceebf8bc None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:18.808260 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa14ee-924c-4606-a63a-0eccceebf8bc None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:18.809012 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa14ee-924c-4606-a63a-0eccceebf8bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:18.809459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa14ee-924c-4606-a63a-0eccceebf8bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:18.810968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa14ee-924c-4606-a63a-0eccceebf8bc None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:18.818733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b2ab2-6920-42e9-856d-3ad2940ea5b6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:18.825314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 309/611] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:18.830711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:18.834864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa14ee-924c-4606-a63a-0eccceebf8bc None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:18.858763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558568a-8d4f-4c97-953b-97554ad94c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:18.859775 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558568a-8d4f-4c97-953b-97554ad94c82 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:18.860253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558568a-8d4f-4c97-953b-97554ad94c82 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:18.860750 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558568a-8d4f-4c97-953b-97554ad94c82 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:18.861657 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558568a-8d4f-4c97-953b-97554ad94c82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:18.862121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558568a-8d4f-4c97-953b-97554ad94c82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:18.863824 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558568a-8d4f-4c97-953b-97554ad94c82 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:18.865757 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:18.878749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 303/612] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:18.884903 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558568a-8d4f-4c97-953b-97554ad94c82 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:19.014554 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558568a-8d4f-4c97-953b-97554ad94c82 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:19.055296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa14ee-924c-4606-a63a-0eccceebf8bc None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:19.056833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05438450-a7e4-4f32-bdd1-9c48401181d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:19.057817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05438450-a7e4-4f32-bdd1-9c48401181d1 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:19.058292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05438450-a7e4-4f32-bdd1-9c48401181d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:19.058739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05438450-a7e4-4f32-bdd1-9c48401181d1 None admin] PATH_INFO: `/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/cdbbaa64ad4844a0b2698dee82f95f23/roles/44e187abae1d4b248fd145ad9cb97ed2` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:19.075248 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05438450-a7e4-4f32-bdd1-9c48401181d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb2bd53d40841eeb1653bd5927fb189, user_id=cdbbaa64ad4844a0b2698dee82f95f23, role_id=44e187abae1d4b248fd145ad9cb97ed2)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:19.077080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05438450-a7e4-4f32-bdd1-9c48401181d1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:19.109024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a4bfe-2c97-4fea-bc76-51b50d46150c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:19.110225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a4bfe-2c97-4fea-bc76-51b50d46150c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:19.111268 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a4bfe-2c97-4fea-bc76-51b50d46150c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:19.111713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a4bfe-2c97-4fea-bc76-51b50d46150c None admin] PATH_INFO: `/v3/users/swift` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:19.120356 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a4bfe-2c97-4fea-bc76-51b50d46150c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:19.121742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a4bfe-2c97-4fea-bc76-51b50d46150c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:19.135555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-5b7a4bfe-2c97-4fea-bc76-51b50d46150c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:19.135555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:19.135555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:19.135555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:19.135555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:19.135555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:19.135555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:19.135555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:19.135555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:19.135555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:19.135555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:19.135555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:19.135555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:19.135555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:19.135555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:19.135555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:19.136547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:19.137474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:19.137474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:19.137474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:19.137474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:19.137474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:19.137474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:19.137474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:19.137474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:19.137474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:19.137474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:19.137474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 310/613] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:19.142390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a4bfe-2c97-4fea-bc76-51b50d46150c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:19.159369 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 304/614] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:18 2022] PUT /identity/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/cdbbaa64ad4844a0b2698dee82f95f23/roles/44e187abae1d4b248fd145ad9cb97ed2 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:38:19.255212 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:19.289736 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a4bfe-2c97-4fea-bc76-51b50d46150c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:19.310377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fe596-263a-4c16-80fb-03acb86ab856 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:19.311020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fe596-263a-4c16-80fb-03acb86ab856 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:19.311493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fe596-263a-4c16-80fb-03acb86ab856 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:19.313316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 305/615] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:19.319604 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df45a3c-2e09-4263-9875-591e3f552f03 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:19.320316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df45a3c-2e09-4263-9875-591e3f552f03 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:19.320784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df45a3c-2e09-4263-9875-591e3f552f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:19.343674 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0df45a3c-2e09-4263-9875-591e3f552f03 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:19.345334 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0df45a3c-2e09-4263-9875-591e3f552f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:19.350093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a0659-b88a-4847-842a-69af1f15da13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:19.350721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a0659-b88a-4847-842a-69af1f15da13 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:19.351050 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a0659-b88a-4847-842a-69af1f15da13 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:19.351320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a0659-b88a-4847-842a-69af1f15da13 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:19.352063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a0659-b88a-4847-842a-69af1f15da13 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:19.352509 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a0659-b88a-4847-842a-69af1f15da13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:19.354150 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a0659-b88a-4847-842a-69af1f15da13 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:19.362681 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:19.388333 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 311/616] 10.222.0.91 () {56 vars in 1135 bytes} [Thu Sep 8 13:38:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:19.394299 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a0659-b88a-4847-842a-69af1f15da13 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:19.529992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 306/617] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:19.611498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a0659-b88a-4847-842a-69af1f15da13 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:19.644124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:19.655179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7f7d8-2109-43da-8ab0-c4ae1a4c1aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:19.655941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7f7d8-2109-43da-8ab0-c4ae1a4c1aee None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:19.656219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7f7d8-2109-43da-8ab0-c4ae1a4c1aee None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:19.656493 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7f7d8-2109-43da-8ab0-c4ae1a4c1aee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:19.667480 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e7f7d8-2109-43da-8ab0-c4ae1a4c1aee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:19.668939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e7f7d8-2109-43da-8ab0-c4ae1a4c1aee None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:19.692943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-09e7f7d8-2109-43da-8ab0-c4ae1a4c1aee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:19.692943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:19.692943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:19.692943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:19.692943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:19.692943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:19.692943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:19.692943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:19.692943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:19.692943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:19.692943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:19.692943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:19.692943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:19.692943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:19.692943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:19.692943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:19.694295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:19.695498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:19.695498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:19.695498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:19.695498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 312/618] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:19.701430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df45a3c-2e09-4263-9875-591e3f552f03 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:19.704921 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae854b-0011-41ce-88ec-3778570b7c74 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:19.705481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae854b-0011-41ce-88ec-3778570b7c74 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:19.705754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae854b-0011-41ce-88ec-3778570b7c74 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:19.710937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 313/619] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:19.712574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35625557-7d9f-4e6b-aae2-3eb80131077e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:19.712859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35625557-7d9f-4e6b-aae2-3eb80131077e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:19.713134 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35625557-7d9f-4e6b-aae2-3eb80131077e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:19.745671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-35625557-7d9f-4e6b-aae2-3eb80131077e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:19.747952 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-35625557-7d9f-4e6b-aae2-3eb80131077e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:19.861353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 314/620] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:19.885964 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35625557-7d9f-4e6b-aae2-3eb80131077e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:19.913893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df45a3c-2e09-4263-9875-591e3f552f03 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:19.989425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650b16c-e258-4d30-9be0-f9937db3cb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:19.990419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650b16c-e258-4d30-9be0-f9937db3cb1f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:19.990911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650b16c-e258-4d30-9be0-f9937db3cb1f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:19.991380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650b16c-e258-4d30-9be0-f9937db3cb1f None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:19.992236 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650b16c-e258-4d30-9be0-f9937db3cb1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:19.993791 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650b16c-e258-4d30-9be0-f9937db3cb1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:19.995440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650b16c-e258-4d30-9be0-f9937db3cb1f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:20.019021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 307/621] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:20.021822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650b16c-e258-4d30-9be0-f9937db3cb1f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:20.035000 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:20.061115 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35625557-7d9f-4e6b-aae2-3eb80131077e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:20.099276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:20.122907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c226b-13d9-43a2-9f48-0c8468730506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:20.125288 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c226b-13d9-43a2-9f48-0c8468730506 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.125791 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c226b-13d9-43a2-9f48-0c8468730506 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.127042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c226b-13d9-43a2-9f48-0c8468730506 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.147324 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c226b-13d9-43a2-9f48-0c8468730506 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:20.155651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-fc2c226b-13d9-43a2-9f48-0c8468730506 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:20.155651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:20.155651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:20.155651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:20.155651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:20.155651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:20.155651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:20.155651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:20.155651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:20.155651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:20.155651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:20.155651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:20.155651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:20.155651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:20.155651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:20.155651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:20.156622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:20.156622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:20.156622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:20.156622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:20.156622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:20.156622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:20.156622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:20.156622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:20.156622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:20.156622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:20.156622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:20.156622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:20.156622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:20.156622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:20.156622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:20.156622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:20.157426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:20.157426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:20.157426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:20.157426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:20.157426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:20.157426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:20.157426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:20.157426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:20.157426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:20.157426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:20.157426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:20.157426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:20.160062 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 315/622] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:20.161168 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c226b-13d9-43a2-9f48-0c8468730506 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:20.203172 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650b16c-e258-4d30-9be0-f9937db3cb1f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:20.242253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f279a-34b7-4a40-813d-4ddb432e33dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:20.242881 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f279a-34b7-4a40-813d-4ddb432e33dd None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.243177 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f279a-34b7-4a40-813d-4ddb432e33dd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.244023 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f279a-34b7-4a40-813d-4ddb432e33dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.252815 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f279a-34b7-4a40-813d-4ddb432e33dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:20.254779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f279a-34b7-4a40-813d-4ddb432e33dd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:20.268428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-ee8f279a-34b7-4a40-813d-4ddb432e33dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:20.268428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:20.268428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:20.268428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:20.268428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:20.268428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:20.268428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:20.268428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:20.268428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:20.268428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:20.268428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:20.268428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:20.268428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:20.268428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:20.268428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:20.269425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:20.270506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:20.270506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:20.270506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:20.270506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:20.270506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:20.270506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:20.270506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:20.270506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:20.270506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 308/623] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:20.271040 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f279a-34b7-4a40-813d-4ddb432e33dd None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:20.286465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c226b-13d9-43a2-9f48-0c8468730506 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:20.327980 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8ffdf-c40f-499c-8672-bf8d80392ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:20.329578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8ffdf-c40f-499c-8672-bf8d80392ff3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.330802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8ffdf-c40f-499c-8672-bf8d80392ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.331578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8ffdf-c40f-499c-8672-bf8d80392ff3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.345768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8ffdf-c40f-499c-8672-bf8d80392ff3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:20.350519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:20.354034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-ecd8ffdf-c40f-499c-8672-bf8d80392ff3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:20.354034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:20.354034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:20.354034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:20.354034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:20.354034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:20.354034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:20.354034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:20.354034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:20.354034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:20.354034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:20.354034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:20.354034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:20.354034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:20.354034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:20.354034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:20.355016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:20.355016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:20.355016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:20.355016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:20.355016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:20.355016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:20.355016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:20.355016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:20.355016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:20.355016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:20.355016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:20.355016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:20.355016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:20.355016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:20.355016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:20.355016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:20.356004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:20.356004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:20.356004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:20.356004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:20.356004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:20.356004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:20.356004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:20.356004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:20.356004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:20.356004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:20.356004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:20.356004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:20.358968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 316/624] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:20.360034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8ffdf-c40f-499c-8672-bf8d80392ff3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:20.404072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:20.412341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f279a-34b7-4a40-813d-4ddb432e33dd None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:20.454501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fedf6c-6efb-4bf5-9bd4-bc1ebab2d4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:20.456175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fedf6c-6efb-4bf5-9bd4-bc1ebab2d4c4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.456648 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fedf6c-6efb-4bf5-9bd4-bc1ebab2d4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.457080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fedf6c-6efb-4bf5-9bd4-bc1ebab2d4c4 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.457893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fedf6c-6efb-4bf5-9bd4-bc1ebab2d4c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:20.458375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fedf6c-6efb-4bf5-9bd4-bc1ebab2d4c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:20.460109 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fedf6c-6efb-4bf5-9bd4-bc1ebab2d4c4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:20.471819 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 309/625] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:20.473051 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fedf6c-6efb-4bf5-9bd4-bc1ebab2d4c4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:20.482665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8ffdf-c40f-499c-8672-bf8d80392ff3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:20.525120 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84009fda-0219-4086-821e-ef6403384a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:20.525592 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84009fda-0219-4086-821e-ef6403384a2b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.525962 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84009fda-0219-4086-821e-ef6403384a2b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.526333 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84009fda-0219-4086-821e-ef6403384a2b None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.527334 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84009fda-0219-4086-821e-ef6403384a2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:20.527535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84009fda-0219-4086-821e-ef6403384a2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:20.529099 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84009fda-0219-4086-821e-ef6403384a2b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:20.542097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 317/626] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:20.543275 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84009fda-0219-4086-821e-ef6403384a2b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:20.610535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fedf6c-6efb-4bf5-9bd4-bc1ebab2d4c4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:20.638878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:20.651462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f47c7-1724-4baa-ab20-b0ea6f8669a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:20.669476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f47c7-1724-4baa-ab20-b0ea6f8669a0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.669476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f47c7-1724-4baa-ab20-b0ea6f8669a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.669476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f47c7-1724-4baa-ab20-b0ea6f8669a0 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.669476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f47c7-1724-4baa-ab20-b0ea6f8669a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:20.669476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f47c7-1724-4baa-ab20-b0ea6f8669a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:20.669476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f47c7-1724-4baa-ab20-b0ea6f8669a0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:20.670044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 310/627] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:20.670698 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f47c7-1724-4baa-ab20-b0ea6f8669a0 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:20.699671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:20.710839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84009fda-0219-4086-821e-ef6403384a2b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:20.747056 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b9a7f-0db8-4c04-b486-b7d919173f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:20.748009 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b9a7f-0db8-4c04-b486-b7d919173f23 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.748131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b9a7f-0db8-4c04-b486-b7d919173f23 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.748349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b9a7f-0db8-4c04-b486-b7d919173f23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.759503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b9a7f-0db8-4c04-b486-b7d919173f23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:20.760952 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b9a7f-0db8-4c04-b486-b7d919173f23 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:20.776353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-989b9a7f-0db8-4c04-b486-b7d919173f23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:20.776353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:20.776353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:20.776353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:20.776353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:20.776353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:20.776353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:20.776353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:20.776353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:20.776353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:20.776353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:20.776353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:20.776353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:20.776353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:20.776353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:20.776353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:20.777473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:20.778408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:20.778408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:20.778408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:20.782142 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 318/628] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:20.783054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b9a7f-0db8-4c04-b486-b7d919173f23 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:20.819012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f47c7-1724-4baa-ab20-b0ea6f8669a0 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:20.860557 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcf4a8a-828f-4029-b6e9-a715ef212558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:20.862198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf4a8a-828f-4029-b6e9-a715ef212558 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.862549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf4a8a-828f-4029-b6e9-a715ef212558 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.862916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcf4a8a-828f-4029-b6e9-a715ef212558 None admin] PATH_INFO: `/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/b3b5ae758fe847adaae0edaa0be095c9/roles/44e187abae1d4b248fd145ad9cb97ed2` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.883200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf4a8a-828f-4029-b6e9-a715ef212558 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb2bd53d40841eeb1653bd5927fb189, user_id=b3b5ae758fe847adaae0edaa0be095c9, role_id=44e187abae1d4b248fd145ad9cb97ed2)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:20.884939 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcf4a8a-828f-4029-b6e9-a715ef212558 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:20.912238 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b9a7f-0db8-4c04-b486-b7d919173f23 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:20.951078 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301eab41-db71-4ad3-a7a7-c0701481fae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:20.951587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301eab41-db71-4ad3-a7a7-c0701481fae5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.951863 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301eab41-db71-4ad3-a7a7-c0701481fae5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.952131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301eab41-db71-4ad3-a7a7-c0701481fae5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.964109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301eab41-db71-4ad3-a7a7-c0701481fae5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:20.965586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301eab41-db71-4ad3-a7a7-c0701481fae5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:20.972542 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:20.980927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-301eab41-db71-4ad3-a7a7-c0701481fae5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:20.980927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:20.980927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:20.980927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:20.980927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:20.980927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:20.980927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:20.980927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:20.980927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:20.980927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:20.980927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:20.980927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:20.980927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:20.980927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:20.980927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:20.980927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:20.981862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:20.982732 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:20.982732 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:20.982732 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:20.982732 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 319/629] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:20.983795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301eab41-db71-4ad3-a7a7-c0701481fae5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:20.989412 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 311/630] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:20 2022] PUT /identity/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/b3b5ae758fe847adaae0edaa0be095c9/roles/44e187abae1d4b248fd145ad9cb97ed2 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:20.997684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fceaf34-adf5-4be3-b521-b732fbf8c203 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.997974 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fceaf34-adf5-4be3-b521-b732fbf8c203 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.998247 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fceaf34-adf5-4be3-b521-b732fbf8c203 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.999799 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 312/631] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:21.009621 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fceaf34-adf5-4be3-b521-b732fbf8c203 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.085338 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:21.135195 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301eab41-db71-4ad3-a7a7-c0701481fae5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:21.190958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fceaf34-adf5-4be3-b521-b732fbf8c203 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:21.205767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca19954-c893-4003-9774-1fedb3c46b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:21.206290 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca19954-c893-4003-9774-1fedb3c46b25 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.206570 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca19954-c893-4003-9774-1fedb3c46b25 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.206841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca19954-c893-4003-9774-1fedb3c46b25 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.207523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca19954-c893-4003-9774-1fedb3c46b25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:21.207826 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca19954-c893-4003-9774-1fedb3c46b25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:21.209276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca19954-c893-4003-9774-1fedb3c46b25 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:21.222692 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 320/632] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:21.225737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0f880-2326-4a11-98da-c56e302e6815 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.225737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0f880-2326-4a11-98da-c56e302e6815 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.225737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0f880-2326-4a11-98da-c56e302e6815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.243358 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-48a0f880-2326-4a11-98da-c56e302e6815 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:21.245019 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-48a0f880-2326-4a11-98da-c56e302e6815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:21.245431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251296f-96b8-4242-b20a-efa5f531680d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:21.246392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251296f-96b8-4242-b20a-efa5f531680d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.247042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251296f-96b8-4242-b20a-efa5f531680d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.247331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251296f-96b8-4242-b20a-efa5f531680d None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.248567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251296f-96b8-4242-b20a-efa5f531680d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:21.250305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251296f-96b8-4242-b20a-efa5f531680d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:21.252890 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251296f-96b8-4242-b20a-efa5f531680d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:21.272033 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 313/633] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:21.274322 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251296f-96b8-4242-b20a-efa5f531680d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.317646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:21.336061 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 321/634] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:21.339112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0f880-2326-4a11-98da-c56e302e6815 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.458914 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:21.473986 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251296f-96b8-4242-b20a-efa5f531680d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:21.504506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0f880-2326-4a11-98da-c56e302e6815 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:21.557743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68b141-37a2-4f36-9cda-fc647981ac49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:21.559005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68b141-37a2-4f36-9cda-fc647981ac49 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.559285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68b141-37a2-4f36-9cda-fc647981ac49 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.559555 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68b141-37a2-4f36-9cda-fc647981ac49 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.575588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975b9e1-b65b-4d83-967a-010998fb07b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:21.576226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975b9e1-b65b-4d83-967a-010998fb07b4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.576506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975b9e1-b65b-4d83-967a-010998fb07b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.576774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975b9e1-b65b-4d83-967a-010998fb07b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.577756 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68b141-37a2-4f36-9cda-fc647981ac49 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:21.579240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68b141-37a2-4f36-9cda-fc647981ac49 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:21.593898 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0975b9e1-b65b-4d83-967a-010998fb07b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:21.595421 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0975b9e1-b65b-4d83-967a-010998fb07b4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:21.601355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-ff68b141-37a2-4f36-9cda-fc647981ac49 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:38:21.601355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:21.601355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:21.601355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:21.601355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:21.601355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:21.601355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:21.601355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:21.601355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:21.601355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:21.601355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:21.601355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:21.601355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:21.601355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:21.601355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:21.601355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:21.602326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:21.603320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:21.603320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:21.603320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:21.603320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:21.603320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:21.603320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:21.603320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:21.603320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:21.603320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:38:21.603320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:21.608016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 314/635] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:21.613953 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68b141-37a2-4f36-9cda-fc647981ac49 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.617130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-0975b9e1-b65b-4d83-967a-010998fb07b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:21.617130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:21.617130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:21.617130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:21.617130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:21.617130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:21.617130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:21.617130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:21.617130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:21.617130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:21.617130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:21.617130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:21.617130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:21.617130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:21.617130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:21.617130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:21.618075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:21.619126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:21.619126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:21.619126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:21.619126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:21.619126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:21.619126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:21.619126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:21.619126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:21.619126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:21.619126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:21.619126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 322/636] 10.222.0.91 () {56 vars in 1093 bytes} [Thu Sep 8 13:38:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:21.624586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975b9e1-b65b-4d83-967a-010998fb07b4 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.699953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:21.761157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975b9e1-b65b-4d83-967a-010998fb07b4 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:21.784538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68b141-37a2-4f36-9cda-fc647981ac49 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:21.827848 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f35607-b454-4363-bfc3-071b23082455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:21.828487 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f35607-b454-4363-bfc3-071b23082455 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.828765 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f35607-b454-4363-bfc3-071b23082455 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.829038 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f35607-b454-4363-bfc3-071b23082455 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.830727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f35607-b454-4363-bfc3-071b23082455 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:21.831107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f35607-b454-4363-bfc3-071b23082455 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:21.832617 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f35607-b454-4363-bfc3-071b23082455 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:21.837326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb438488-ba74-435c-8825-ba2467b2ba3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:21.838594 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb438488-ba74-435c-8825-ba2467b2ba3b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.839260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb438488-ba74-435c-8825-ba2467b2ba3b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.839802 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb438488-ba74-435c-8825-ba2467b2ba3b None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.840657 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb438488-ba74-435c-8825-ba2467b2ba3b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:21.841271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb438488-ba74-435c-8825-ba2467b2ba3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:21.842989 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb438488-ba74-435c-8825-ba2467b2ba3b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:21.853450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 323/637] 10.222.0.91 () {56 vars in 1133 bytes} [Thu Sep 8 13:38:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:21.854857 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f35607-b454-4363-bfc3-071b23082455 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.871007 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 315/638] 10.222.0.91 () {56 vars in 1137 bytes} [Thu Sep 8 13:38:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:21.871982 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb438488-ba74-435c-8825-ba2467b2ba3b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.893737 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:22.013442 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f35607-b454-4363-bfc3-071b23082455 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:22.024262 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:22.024616 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb438488-ba74-435c-8825-ba2467b2ba3b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:22.071371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a00524-7813-4aca-9fd4-286fe602f396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:22.072263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a00524-7813-4aca-9fd4-286fe602f396 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.072539 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a00524-7813-4aca-9fd4-286fe602f396 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.072968 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a00524-7813-4aca-9fd4-286fe602f396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:22.077246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b77882-6e87-414a-b333-84a39864faf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:22.077986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b77882-6e87-414a-b333-84a39864faf7 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.078364 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b77882-6e87-414a-b333-84a39864faf7 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.078663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b77882-6e87-414a-b333-84a39864faf7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:22.101830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b77882-6e87-414a-b333-84a39864faf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:22.106613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-85b77882-6e87-414a-b333-84a39864faf7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:22.106613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:22.106613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:22.106613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:22.106613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:22.106613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:22.106613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:22.106613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:22.106613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:22.106613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:22.106613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:22.106613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:22.106613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:22.106613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:22.106613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:22.106613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:22.108184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:22.108184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:22.108184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:22.108184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:22.108184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:22.108184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:22.108184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:22.108184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:22.108184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:22.108184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:22.108184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:22.108184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:22.108184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:22.108184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:22.108184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:22.108184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:22.109556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:22.109556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:22.109556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:22.109556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:22.109556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:22.109556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:22.109556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:22.109556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:22.109556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:22.109556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:22.109556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:22.109556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:22.109556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 324/639] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:22.109556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b77882-6e87-414a-b333-84a39864faf7 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:22.179855 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a00524-7813-4aca-9fd4-286fe602f396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:22.181303 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a00524-7813-4aca-9fd4-286fe602f396 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:22.197196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:22.245049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b77882-6e87-414a-b333-84a39864faf7 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:22.297376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb124b-ada3-42b1-92eb-acaf32c67d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:22.298494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb124b-ada3-42b1-92eb-acaf32c67d9e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.299006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb124b-ada3-42b1-92eb-acaf32c67d9e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.299481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb124b-ada3-42b1-92eb-acaf32c67d9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:22.344861 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 316/640] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:22.344861 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a9f12-f3f0-47a7-8b37-89ecabde0723 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.344861 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a9f12-f3f0-47a7-8b37-89ecabde0723 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.344861 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a9f12-f3f0-47a7-8b37-89ecabde0723 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:22.344861 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 317/641] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:22.344861 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a9f12-f3f0-47a7-8b37-89ecabde0723 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:22.365942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:22.415002 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb124b-ada3-42b1-92eb-acaf32c67d9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:22.417403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb124b-ada3-42b1-92eb-acaf32c67d9e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:22.500222 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a9f12-f3f0-47a7-8b37-89ecabde0723 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:22.526416 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:22.556067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1419b-5707-43d7-ae1c-962b476e936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:22.557965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1419b-5707-43d7-ae1c-962b476e936c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.558143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1419b-5707-43d7-ae1c-962b476e936c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.558539 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1419b-5707-43d7-ae1c-962b476e936c None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:22.559881 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f1419b-5707-43d7-ae1c-962b476e936c None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:22.560304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f1419b-5707-43d7-ae1c-962b476e936c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:22.563296 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f1419b-5707-43d7-ae1c-962b476e936c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:22.569416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 325/642] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:22.569416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb124b-ada3-42b1-92eb-acaf32c67d9e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:22.576412 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 318/643] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:22.578650 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a0815-96bb-4a20-9488-4d640383c389 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.579192 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a0815-96bb-4a20-9488-4d640383c389 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.579704 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a0815-96bb-4a20-9488-4d640383c389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:22.600106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9a7a0815-96bb-4a20-9488-4d640383c389 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:22.601476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9a7a0815-96bb-4a20-9488-4d640383c389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:22.664763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:22.702923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 319/644] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:22.702923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a0815-96bb-4a20-9488-4d640383c389 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:22.715129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb124b-ada3-42b1-92eb-acaf32c67d9e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:22.764431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d946caa-1986-44d6-ba37-178c0d77c3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:22.765376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d946caa-1986-44d6-ba37-178c0d77c3ad None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.765716 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d946caa-1986-44d6-ba37-178c0d77c3ad None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.765992 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d946caa-1986-44d6-ba37-178c0d77c3ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:22.784159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d946caa-1986-44d6-ba37-178c0d77c3ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:22.790120 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d946caa-1986-44d6-ba37-178c0d77c3ad None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:22.808807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-7d946caa-1986-44d6-ba37-178c0d77c3ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:22.808807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:22.808807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:22.808807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:22.808807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:22.808807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:22.808807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:22.808807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:22.808807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:22.808807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:22.808807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:22.808807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:22.808807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:22.808807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:22.808807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:22.808807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:22.810722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:22.812446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:22.812446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:22.812446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:22.814914 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 326/645] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:22.815867 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d946caa-1986-44d6-ba37-178c0d77c3ad None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:22.852485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a0815-96bb-4a20-9488-4d640383c389 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:22.870176 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:22.924063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369700e-5669-4616-a554-87d633356edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:22.925068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369700e-5669-4616-a554-87d633356edf None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.925547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369700e-5669-4616-a554-87d633356edf None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.926212 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369700e-5669-4616-a554-87d633356edf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:22.946195 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d946caa-1986-44d6-ba37-178c0d77c3ad None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:22.946767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369700e-5669-4616-a554-87d633356edf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:22.949304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369700e-5669-4616-a554-87d633356edf None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:22.968588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-9369700e-5669-4616-a554-87d633356edf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:22.968588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:22.968588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:22.968588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:22.968588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:22.968588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:22.968588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:22.968588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:22.968588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:22.968588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:22.968588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:22.968588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:22.968588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:22.968588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:22.968588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:22.968588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:22.969673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:22.971317 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:22.971317 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:22.971317 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:22.971770 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:22.974936 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 320/646] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:22.976035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369700e-5669-4616-a554-87d633356edf None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:22.992198 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6a9f0-ced6-4786-853f-1d2b2c8d4f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:22.993148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6a9f0-ced6-4786-853f-1d2b2c8d4f60 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.994391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6a9f0-ced6-4786-853f-1d2b2c8d4f60 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.995680 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6a9f0-ced6-4786-853f-1d2b2c8d4f60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:23.008262 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6a9f0-ced6-4786-853f-1d2b2c8d4f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:23.009995 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6a9f0-ced6-4786-853f-1d2b2c8d4f60 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:23.029409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-f5b6a9f0-ced6-4786-853f-1d2b2c8d4f60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:23.029409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:23.029409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:23.029409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:23.029409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:23.029409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:23.029409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:23.029409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:23.029409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:23.029409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:23.029409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:23.029409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:23.029409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:23.029409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:23.029409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:23.029409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:23.031202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:23.032763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:23.032763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:23.032763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:23.036306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 327/647] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:23.037185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6a9f0-ced6-4786-853f-1d2b2c8d4f60 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:23.113974 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369700e-5669-4616-a554-87d633356edf None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:23.150185 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34ffd1-a0ad-4c9b-bf4c-4622a562fe98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:23.150802 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34ffd1-a0ad-4c9b-bf4c-4622a562fe98 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:23.151087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34ffd1-a0ad-4c9b-bf4c-4622a562fe98 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:23.151357 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34ffd1-a0ad-4c9b-bf4c-4622a562fe98 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:23.151984 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34ffd1-a0ad-4c9b-bf4c-4622a562fe98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:23.152340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34ffd1-a0ad-4c9b-bf4c-4622a562fe98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:23.153692 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34ffd1-a0ad-4c9b-bf4c-4622a562fe98 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:23.171945 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 321/648] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:23.172629 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34ffd1-a0ad-4c9b-bf4c-4622a562fe98 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:23.181724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6a9f0-ced6-4786-853f-1d2b2c8d4f60 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:23.194453 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:23.245900 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57a251-f28a-4b5e-a647-22d5d8d59120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:23.246736 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57a251-f28a-4b5e-a647-22d5d8d59120 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:23.247208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57a251-f28a-4b5e-a647-22d5d8d59120 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:23.247665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57a251-f28a-4b5e-a647-22d5d8d59120 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:23.248741 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57a251-f28a-4b5e-a647-22d5d8d59120 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:23.249188 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57a251-f28a-4b5e-a647-22d5d8d59120 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:23.251888 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57a251-f28a-4b5e-a647-22d5d8d59120 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:23.276139 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 328/649] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:23.276139 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57a251-f28a-4b5e-a647-22d5d8d59120 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:23.306165 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:23.325782 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34ffd1-a0ad-4c9b-bf4c-4622a562fe98 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:23.362822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304436c-38b1-4635-813d-14dc8f72a4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:23.363479 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304436c-38b1-4635-813d-14dc8f72a4b9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:23.363753 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304436c-38b1-4635-813d-14dc8f72a4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:23.364069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304436c-38b1-4635-813d-14dc8f72a4b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:23.375712 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304436c-38b1-4635-813d-14dc8f72a4b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:23.378613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57a251-f28a-4b5e-a647-22d5d8d59120 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:23.380489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-1304436c-38b1-4635-813d-14dc8f72a4b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:23.380489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:23.380489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:23.380489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:23.380489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:23.380489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:23.380489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:23.380489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:23.380489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:23.380489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:23.380489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:23.380489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:23.380489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:23.380489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:23.380489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:23.380489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:23.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:23.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:23.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:23.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:23.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:23.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:23.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:23.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:23.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:23.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:23.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:23.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:23.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:23.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:23.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:23.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:23.383615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:23.383615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:23.383615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:23.383615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:23.383615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:23.383615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:23.383615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:23.383615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:23.383615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:23.383615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:23.383615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:23.383615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:23.383615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 322/650] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:23.383615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304436c-38b1-4635-813d-14dc8f72a4b9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:23.416762 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b01b9e-39c4-4a57-9e55-e84fecf6d98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:23.417368 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b01b9e-39c4-4a57-9e55-e84fecf6d98f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:23.417644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b01b9e-39c4-4a57-9e55-e84fecf6d98f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:23.417918 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b01b9e-39c4-4a57-9e55-e84fecf6d98f None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:23.418555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b01b9e-39c4-4a57-9e55-e84fecf6d98f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:23.418855 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b01b9e-39c4-4a57-9e55-e84fecf6d98f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:23.420267 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b01b9e-39c4-4a57-9e55-e84fecf6d98f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:23.431990 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 329/651] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:23.432793 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b01b9e-39c4-4a57-9e55-e84fecf6d98f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:23.468409 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:23.491146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304436c-38b1-4635-813d-14dc8f72a4b9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:23.524815 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d773bc-8bf7-444b-9e1b-62a4687bf257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:23.525436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d773bc-8bf7-444b-9e1b-62a4687bf257 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:23.525709 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d773bc-8bf7-444b-9e1b-62a4687bf257 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:23.525976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d773bc-8bf7-444b-9e1b-62a4687bf257 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:23.529297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:23.536943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d773bc-8bf7-444b-9e1b-62a4687bf257 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:23.538577 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d773bc-8bf7-444b-9e1b-62a4687bf257 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:23.539160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b01b9e-39c4-4a57-9e55-e84fecf6d98f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:23.550839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-d5d773bc-8bf7-444b-9e1b-62a4687bf257 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:23.550839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:23.550839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:23.550839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:23.550839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:23.550839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:23.550839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:23.550839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:23.550839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:23.550839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:23.550839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:23.550839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:23.550839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:23.550839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:23.550839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:23.551769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:23.552637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:23.552637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:23.552637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:23.552637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:23.552637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:23.552637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:23.552637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:23.552637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:23.552637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 323/652] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:23.553167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aaf616-7884-4b6c-ad42-e46b24615ae2 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:23.553595 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aaf616-7884-4b6c-ad42-e46b24615ae2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:23.554037 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aaf616-7884-4b6c-ad42-e46b24615ae2 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:23.555589 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 324/653] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:23.556308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aaf616-7884-4b6c-ad42-e46b24615ae2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:23.577367 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c12f0-33bf-46b9-a4b7-173d1cf0ef6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:23.577993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c12f0-33bf-46b9-a4b7-173d1cf0ef6e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:23.578271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c12f0-33bf-46b9-a4b7-173d1cf0ef6e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:23.578544 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c12f0-33bf-46b9-a4b7-173d1cf0ef6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:23.585880 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c12f0-33bf-46b9-a4b7-173d1cf0ef6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:23.587359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c12f0-33bf-46b9-a4b7-173d1cf0ef6e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:23.598332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-9f6c12f0-33bf-46b9-a4b7-173d1cf0ef6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:23.598332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:23.598332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:23.598332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:23.598332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:23.598332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:23.598332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:23.598332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:23.598332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:23.598332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:23.598332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:23.598332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:23.598332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:23.598332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:23.598332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:23.599377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:23.605465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:23.605465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:23.605465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:23.605465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:23.605465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:23.605465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:23.605465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:23.605465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:23.605465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 330/654] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:23.605465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c12f0-33bf-46b9-a4b7-173d1cf0ef6e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:23.671831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aaf616-7884-4b6c-ad42-e46b24615ae2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:23.715577 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d0296-ab7b-41fc-b85e-1f4ab324b01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:23.716531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d0296-ab7b-41fc-b85e-1f4ab324b01a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:23.716992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d0296-ab7b-41fc-b85e-1f4ab324b01a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:23.717455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d0296-ab7b-41fc-b85e-1f4ab324b01a None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:23.718565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d0296-ab7b-41fc-b85e-1f4ab324b01a None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:23.719227 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d0296-ab7b-41fc-b85e-1f4ab324b01a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:23.722022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d0296-ab7b-41fc-b85e-1f4ab324b01a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:23.729107 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:23.729887 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c12f0-33bf-46b9-a4b7-173d1cf0ef6e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:23.733116 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 325/655] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:23.733919 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d0296-ab7b-41fc-b85e-1f4ab324b01a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:23.773219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1832df7-1407-4865-992e-9004069a2e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:23.773966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1832df7-1407-4865-992e-9004069a2e8d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:23.774282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1832df7-1407-4865-992e-9004069a2e8d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:23.774599 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1832df7-1407-4865-992e-9004069a2e8d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:23.782310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1832df7-1407-4865-992e-9004069a2e8d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:23.784342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1832df7-1407-4865-992e-9004069a2e8d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:23.796063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-d1832df7-1407-4865-992e-9004069a2e8d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:38:23.796063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:23.796063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:23.796063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:23.796063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:23.796063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:23.796063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:23.796063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:23.796063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:23.796063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:23.796063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:23.796063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:23.796063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:23.796063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:23.796063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:23.796063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:23.797018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:23.798058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:23.798058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:23.798058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:23.798058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:23.798058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:23.798058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:23.798058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:23.798058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:23.798058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:38:23.798058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:23.798058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 331/656] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:23.799217 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ccaff-9e6d-467f-98b7-e49ebb3032e1 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:23.799217 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ccaff-9e6d-467f-98b7-e49ebb3032e1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:23.799528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ccaff-9e6d-467f-98b7-e49ebb3032e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:23.819017 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ed6ccaff-9e6d-467f-98b7-e49ebb3032e1 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:23.820223 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ed6ccaff-9e6d-467f-98b7-e49ebb3032e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:23.823058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:23.856350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d0296-ab7b-41fc-b85e-1f4ab324b01a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:23.887529 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 332/657] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:23.888385 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ccaff-9e6d-467f-98b7-e49ebb3032e1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:23.904567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e65bb3-70c2-46b1-8568-8df75ec2e3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:23.905180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e65bb3-70c2-46b1-8568-8df75ec2e3e4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:23.905457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e65bb3-70c2-46b1-8568-8df75ec2e3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:23.905727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e65bb3-70c2-46b1-8568-8df75ec2e3e4 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:23.906418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e65bb3-70c2-46b1-8568-8df75ec2e3e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:23.906727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e65bb3-70c2-46b1-8568-8df75ec2e3e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:23.908214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e65bb3-70c2-46b1-8568-8df75ec2e3e4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:23.927079 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 326/658] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:23.927079 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e65bb3-70c2-46b1-8568-8df75ec2e3e4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:24.011014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ccaff-9e6d-467f-98b7-e49ebb3032e1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:24.015916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:24.045857 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6999d6f-d0be-4262-86f8-5b66c25cfc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:24.046465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6999d6f-d0be-4262-86f8-5b66c25cfc04 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:24.046742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6999d6f-d0be-4262-86f8-5b66c25cfc04 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:24.047048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6999d6f-d0be-4262-86f8-5b66c25cfc04 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:24.047733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6999d6f-d0be-4262-86f8-5b66c25cfc04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:24.048034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6999d6f-d0be-4262-86f8-5b66c25cfc04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:24.049497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6999d6f-d0be-4262-86f8-5b66c25cfc04 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:24.058075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:24.068011 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 333/659] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:24.068011 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6999d6f-d0be-4262-86f8-5b66c25cfc04 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:24.087498 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e65bb3-70c2-46b1-8568-8df75ec2e3e4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:24.131178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16c832-bb4a-4d08-a57e-fe30b06ffe95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:24.132101 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16c832-bb4a-4d08-a57e-fe30b06ffe95 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:24.132101 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16c832-bb4a-4d08-a57e-fe30b06ffe95 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:24.132376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16c832-bb4a-4d08-a57e-fe30b06ffe95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:24.152055 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16c832-bb4a-4d08-a57e-fe30b06ffe95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:24.153519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16c832-bb4a-4d08-a57e-fe30b06ffe95 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:24.169018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-ca16c832-bb4a-4d08-a57e-fe30b06ffe95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:24.169018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:24.169018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:24.169018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:24.169018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:24.169018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:24.169018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:24.169018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:24.169018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:24.169018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:24.169018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:24.169018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:24.169018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:24.169018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:24.169018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:24.169018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:24.170527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:24.172005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:24.172005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:24.172005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:24.172005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 327/660] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:24.172005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16c832-bb4a-4d08-a57e-fe30b06ffe95 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:24.218473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6999d6f-d0be-4262-86f8-5b66c25cfc04 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:24.252310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b29f69-432f-42ae-9062-0d12a1fc41fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:24.252909 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b29f69-432f-42ae-9062-0d12a1fc41fc None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:24.253184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b29f69-432f-42ae-9062-0d12a1fc41fc None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:24.253452 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b29f69-432f-42ae-9062-0d12a1fc41fc None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:24.254162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b29f69-432f-42ae-9062-0d12a1fc41fc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:24.254515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b29f69-432f-42ae-9062-0d12a1fc41fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:24.255980 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b29f69-432f-42ae-9062-0d12a1fc41fc None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:24.273162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 334/661] 10.222.0.91 () {56 vars in 1137 bytes} [Thu Sep 8 13:38:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:24.273966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b29f69-432f-42ae-9062-0d12a1fc41fc None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:24.286228 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16c832-bb4a-4d08-a57e-fe30b06ffe95 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:24.305454 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:24.330299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbcbdd0-f908-4767-b993-cb6637dd0edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:24.331784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcbdd0-f908-4767-b993-cb6637dd0edb None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:24.331959 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcbdd0-f908-4767-b993-cb6637dd0edb None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:24.332410 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcbdd0-f908-4767-b993-cb6637dd0edb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:24.333631 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcbdd0-f908-4767-b993-cb6637dd0edb None admin] RBAC: Adding query filter params (role.id=44e187abae1d4b248fd145ad9cb97ed2, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=38127f41852d412ea6ad270f1618fce2) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:24.334146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcbdd0-f908-4767-b993-cb6637dd0edb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:24.336705 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcbdd0-f908-4767-b993-cb6637dd0edb None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:24.348042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 328/662] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:24 2022] GET /identity/v3/role_assignments?user.id=38127f41852d412ea6ad270f1618fce2&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=44e187abae1d4b248fd145ad9cb97ed2 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:24.348850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbcbdd0-f908-4767-b993-cb6637dd0edb None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:24.355744 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:24.407069 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b29f69-432f-42ae-9062-0d12a1fc41fc None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:24.455858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df967b-e27b-4232-a8c3-a5ec64afa0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:24.456476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df967b-e27b-4232-a8c3-a5ec64afa0ce None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:24.456749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df967b-e27b-4232-a8c3-a5ec64afa0ce None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:24.457020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df967b-e27b-4232-a8c3-a5ec64afa0ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:24.457749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df967b-e27b-4232-a8c3-a5ec64afa0ce None admin] RBAC: Adding query filter params (role.id=44e187abae1d4b248fd145ad9cb97ed2, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=01b060c70e1b440186eeac37d9c34567) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:24.458054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df967b-e27b-4232-a8c3-a5ec64afa0ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:24.459572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df967b-e27b-4232-a8c3-a5ec64afa0ce None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:24.466165 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 335/663] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:24 2022] GET /identity/v3/role_assignments?user.id=01b060c70e1b440186eeac37d9c34567&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=44e187abae1d4b248fd145ad9cb97ed2 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:24.467045 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df967b-e27b-4232-a8c3-a5ec64afa0ce None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:24.518820 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbcbdd0-f908-4767-b993-cb6637dd0edb None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:24.569072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb293c6b-0894-407b-b368-0b7430790658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:24.569723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb293c6b-0894-407b-b368-0b7430790658 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:24.570017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb293c6b-0894-407b-b368-0b7430790658 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:24.570325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb293c6b-0894-407b-b368-0b7430790658 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:24.584816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb293c6b-0894-407b-b368-0b7430790658 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:24.590829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-eb293c6b-0894-407b-b368-0b7430790658 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:24.590829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:24.590829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:24.590829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:24.590829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:24.590829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:24.590829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:24.590829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:24.590829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:24.590829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:24.590829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:24.590829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:24.590829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:24.590829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:24.590829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:24.590829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:24.591825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:24.591825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:24.591825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:24.591825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:24.591825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:24.591825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:24.591825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:24.591825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:24.591825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:24.591825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:24.591825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:24.591825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:24.591825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:24.591825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:24.591825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:24.591825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:24.592679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:24.592679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:24.592679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:24.592679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:24.592679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:24.592679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:24.592679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:24.592679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:24.592679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:24.592679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:24.592679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:24.592679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:24.593437 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 329/664] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:24.594746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb293c6b-0894-407b-b368-0b7430790658 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:24.635907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df967b-e27b-4232-a8c3-a5ec64afa0ce None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:24.646798 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:24.666243 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:24.684150 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4727d7d-311a-46e7-8e57-9195908022f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:24.684763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4727d7d-311a-46e7-8e57-9195908022f1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:24.685041 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4727d7d-311a-46e7-8e57-9195908022f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:24.685311 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4727d7d-311a-46e7-8e57-9195908022f1 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:24.685950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4727d7d-311a-46e7-8e57-9195908022f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:24.686252 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4727d7d-311a-46e7-8e57-9195908022f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:24.687683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4727d7d-311a-46e7-8e57-9195908022f1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:24.701775 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 336/665] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:24.709386 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4727d7d-311a-46e7-8e57-9195908022f1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:24.731021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb293c6b-0894-407b-b368-0b7430790658 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:24.780342 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158440e-08d9-485e-a1a3-4f81128a7035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:24.780955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158440e-08d9-485e-a1a3-4f81128a7035 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:24.781236 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158440e-08d9-485e-a1a3-4f81128a7035 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:24.781509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158440e-08d9-485e-a1a3-4f81128a7035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:24.836425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4727d7d-311a-46e7-8e57-9195908022f1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:24.862042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158440e-08d9-485e-a1a3-4f81128a7035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:24.863768 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158440e-08d9-485e-a1a3-4f81128a7035 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:24.880210 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f55c7-4448-4641-97b7-bed09fc3e10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:24.881131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f55c7-4448-4641-97b7-bed09fc3e10f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:24.881513 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f55c7-4448-4641-97b7-bed09fc3e10f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:24.881886 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f55c7-4448-4641-97b7-bed09fc3e10f None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:24.883108 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f55c7-4448-4641-97b7-bed09fc3e10f None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:24.883789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f55c7-4448-4641-97b7-bed09fc3e10f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:24.885765 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f55c7-4448-4641-97b7-bed09fc3e10f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:24.897949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 337/666] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:24.899208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f55c7-4448-4641-97b7-bed09fc3e10f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:24.905162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:24.966019 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:24.985070 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 330/667] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:24.986284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158440e-08d9-485e-a1a3-4f81128a7035 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:25.031205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f55c7-4448-4641-97b7-bed09fc3e10f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:25.067821 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ac314-8daf-4dd5-a8c3-3a9cbae849da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:25.068454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ac314-8daf-4dd5-a8c3-3a9cbae849da None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:25.068729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ac314-8daf-4dd5-a8c3-3a9cbae849da None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:25.068997 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ac314-8daf-4dd5-a8c3-3a9cbae849da None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:25.077236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ac314-8daf-4dd5-a8c3-3a9cbae849da None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:25.078687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ac314-8daf-4dd5-a8c3-3a9cbae849da None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:25.091350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-171ac314-8daf-4dd5-a8c3-3a9cbae849da None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:38:25.091350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:25.091350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:25.091350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:25.091350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:25.091350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:25.091350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:25.091350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:25.091350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:25.091350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:25.091350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:25.091350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:25.091350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:25.091350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:25.091350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:25.091350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:25.092315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:25.093258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:25.093258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:25.093258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:25.093258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:25.093258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:25.093258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:25.093258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:25.093258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:25.093258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:38:25.093258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:25.097582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 338/668] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:25.098462 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ac314-8daf-4dd5-a8c3-3a9cbae849da None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:25.122134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158440e-08d9-485e-a1a3-4f81128a7035 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:25.160181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad3432-4116-4ce1-8b9e-7f41513ecbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:25.160908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad3432-4116-4ce1-8b9e-7f41513ecbab None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:25.161183 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad3432-4116-4ce1-8b9e-7f41513ecbab None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:25.161449 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad3432-4116-4ce1-8b9e-7f41513ecbab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:25.164441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:25.174269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad3432-4116-4ce1-8b9e-7f41513ecbab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:25.175764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad3432-4116-4ce1-8b9e-7f41513ecbab None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:25.192625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-cfad3432-4116-4ce1-8b9e-7f41513ecbab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:25.192625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:25.192625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:25.192625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:25.192625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:25.192625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:25.192625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:25.192625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:25.192625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:25.192625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:25.192625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:25.192625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:25.192625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:25.192625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:25.192625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:25.192625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:25.196308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:25.197190 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:25.197190 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:25.197190 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:25.197190 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 331/669] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:25.197190 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad3432-4116-4ce1-8b9e-7f41513ecbab None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:25.232637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ac314-8daf-4dd5-a8c3-3a9cbae849da None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:25.236684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:25.282195 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167fb72-712c-4ce5-806b-26e137c636a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:25.282651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167fb72-712c-4ce5-806b-26e137c636a0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:25.283506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167fb72-712c-4ce5-806b-26e137c636a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:25.283777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167fb72-712c-4ce5-806b-26e137c636a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:25.300134 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167fb72-712c-4ce5-806b-26e137c636a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:25.302800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167fb72-712c-4ce5-806b-26e137c636a0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:25.324306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-6167fb72-712c-4ce5-806b-26e137c636a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:25.324306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:25.324306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:25.324306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:25.324306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:25.324306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:25.324306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:25.324306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:25.324306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:25.324306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:25.324306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:25.324306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:25.324306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:25.324306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:25.324306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:25.324306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:25.325336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:25.326221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:25.326221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:25.326221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:25.329487 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 339/670] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:25.330943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167fb72-712c-4ce5-806b-26e137c636a0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:25.353838 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad3432-4116-4ce1-8b9e-7f41513ecbab None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:25.397435 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3811801-965e-4867-b6e7-35cc3467afbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:25.398041 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3811801-965e-4867-b6e7-35cc3467afbc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:25.398676 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3811801-965e-4867-b6e7-35cc3467afbc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:25.398818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3811801-965e-4867-b6e7-35cc3467afbc None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:25.399524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3811801-965e-4867-b6e7-35cc3467afbc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:25.399979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3811801-965e-4867-b6e7-35cc3467afbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:25.401358 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3811801-965e-4867-b6e7-35cc3467afbc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:25.429534 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 332/671] 10.222.0.91 () {56 vars in 1139 bytes} [Thu Sep 8 13:38:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:25.430521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3811801-965e-4867-b6e7-35cc3467afbc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:25.472132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167fb72-712c-4ce5-806b-26e137c636a0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:25.505914 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:25.542718 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbc256-3a4f-4891-ba78-3192ed29a743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:25.544283 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbc256-3a4f-4891-ba78-3192ed29a743 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:25.546247 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbc256-3a4f-4891-ba78-3192ed29a743 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:25.547147 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbc256-3a4f-4891-ba78-3192ed29a743 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:25.550296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbc256-3a4f-4891-ba78-3192ed29a743 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:25.551928 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbc256-3a4f-4891-ba78-3192ed29a743 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:25.555340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbc256-3a4f-4891-ba78-3192ed29a743 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:25.582986 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3811801-965e-4867-b6e7-35cc3467afbc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:25.586701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 340/672] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:25.588148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbc256-3a4f-4891-ba78-3192ed29a743 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:25.627329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:25.633316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b7ffa-49fb-4dec-97ad-3af915ff2a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:25.635019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b7ffa-49fb-4dec-97ad-3af915ff2a47 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:25.641917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b7ffa-49fb-4dec-97ad-3af915ff2a47 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:25.641917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b7ffa-49fb-4dec-97ad-3af915ff2a47 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:25.641917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b7ffa-49fb-4dec-97ad-3af915ff2a47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:25.641917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b7ffa-49fb-4dec-97ad-3af915ff2a47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:25.641917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b7ffa-49fb-4dec-97ad-3af915ff2a47 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:25.664690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 333/673] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:25.665831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b7ffa-49fb-4dec-97ad-3af915ff2a47 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:25.759306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbc256-3a4f-4891-ba78-3192ed29a743 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:25.792533 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b7ffa-49fb-4dec-97ad-3af915ff2a47 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:25.801417 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:25.805967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5598b4-c8fb-4c49-9d7d-629d4a9bee83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:25.806787 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5598b4-c8fb-4c49-9d7d-629d4a9bee83 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:25.807319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5598b4-c8fb-4c49-9d7d-629d4a9bee83 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:25.807739 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5598b4-c8fb-4c49-9d7d-629d4a9bee83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:25.837529 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12ddb8-038b-461a-857c-d32e45e9442a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:25.838159 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12ddb8-038b-461a-857c-d32e45e9442a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:25.838436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12ddb8-038b-461a-857c-d32e45e9442a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:25.838708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12ddb8-038b-461a-857c-d32e45e9442a None admin] PATH_INFO: `/v3/users/swift` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:25.848893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12ddb8-038b-461a-857c-d32e45e9442a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:25.851074 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12ddb8-038b-461a-857c-d32e45e9442a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:25.865972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-bd12ddb8-038b-461a-857c-d32e45e9442a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:25.865972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:25.865972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:25.865972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:25.865972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:25.865972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:25.865972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:25.865972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:25.865972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:25.865972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:25.865972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:25.865972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:25.865972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:25.865972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:25.865972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:25.865972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:25.867102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:25.868080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:25.868080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:25.868080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:25.868080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:25.868080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:25.868080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:25.868080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:25.868080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:25.868080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:25.868080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:25.871039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 334/674] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:25.872055 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12ddb8-038b-461a-857c-d32e45e9442a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:25.900723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5598b4-c8fb-4c49-9d7d-629d4a9bee83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:25.902570 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5598b4-c8fb-4c49-9d7d-629d4a9bee83 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:25.924709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:25.997767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12ddb8-038b-461a-857c-d32e45e9442a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:26.035421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356983d6-acc2-47d3-8c83-1fb13dbaa1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:26.036457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356983d6-acc2-47d3-8c83-1fb13dbaa1d8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:26.036457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356983d6-acc2-47d3-8c83-1fb13dbaa1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:26.036457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356983d6-acc2-47d3-8c83-1fb13dbaa1d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:26.049342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 341/675] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:26.050655 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5598b4-c8fb-4c49-9d7d-629d4a9bee83 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:26.050854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356983d6-acc2-47d3-8c83-1fb13dbaa1d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:26.053145 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356983d6-acc2-47d3-8c83-1fb13dbaa1d8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:26.071587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-356983d6-acc2-47d3-8c83-1fb13dbaa1d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:26.071587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:26.071587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:26.071587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:26.071587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:26.071587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:26.071587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:26.071587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:26.071587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:26.071587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:26.071587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:26.071587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:26.071587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:26.071587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:26.071587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:26.073255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:26.074310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:26.074310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:26.074310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:26.074310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:26.074310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:26.074310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:26.074310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:26.074310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:26.079027 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 335/676] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:26.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356983d6-acc2-47d3-8c83-1fb13dbaa1d8 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:26.118793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:26.215466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5598b4-c8fb-4c49-9d7d-629d4a9bee83 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:26.238511 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:26.246306 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356983d6-acc2-47d3-8c83-1fb13dbaa1d8 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:26.259794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238e06f-01f0-4d48-968f-16b1a63fe323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:26.260755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238e06f-01f0-4d48-968f-16b1a63fe323 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:26.261238 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238e06f-01f0-4d48-968f-16b1a63fe323 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:26.261657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238e06f-01f0-4d48-968f-16b1a63fe323 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:26.262621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238e06f-01f0-4d48-968f-16b1a63fe323 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:26.263155 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238e06f-01f0-4d48-968f-16b1a63fe323 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:26.265924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238e06f-01f0-4d48-968f-16b1a63fe323 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:26.301589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 342/677] 10.222.0.91 () {56 vars in 1135 bytes} [Thu Sep 8 13:38:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:26.302758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238e06f-01f0-4d48-968f-16b1a63fe323 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:26.309592 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af4ed9-50b8-44e5-bc14-f04a604cb5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:26.310585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af4ed9-50b8-44e5-bc14-f04a604cb5a2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:26.311105 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af4ed9-50b8-44e5-bc14-f04a604cb5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:26.311543 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af4ed9-50b8-44e5-bc14-f04a604cb5a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:26.332482 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af4ed9-50b8-44e5-bc14-f04a604cb5a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:26.334265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af4ed9-50b8-44e5-bc14-f04a604cb5a2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:26.352817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-15af4ed9-50b8-44e5-bc14-f04a604cb5a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:26.352817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:26.352817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:26.352817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:26.352817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:26.352817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:26.352817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:26.352817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:26.352817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:26.352817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:26.352817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:26.352817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:26.352817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:26.352817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:26.352817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:26.352817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:26.354831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:26.356693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:26.356693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:26.356693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:26.358555 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 336/678] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:26.359468 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af4ed9-50b8-44e5-bc14-f04a604cb5a2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:26.452155 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:26.465893 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238e06f-01f0-4d48-968f-16b1a63fe323 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:26.477622 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af4ed9-50b8-44e5-bc14-f04a604cb5a2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:26.516749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21092861-e777-4f57-b8b1-d69b5e0a73ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:26.518173 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21092861-e777-4f57-b8b1-d69b5e0a73ee None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:26.518749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21092861-e777-4f57-b8b1-d69b5e0a73ee None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:26.519284 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21092861-e777-4f57-b8b1-d69b5e0a73ee None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:26.520459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21092861-e777-4f57-b8b1-d69b5e0a73ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:26.520847 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21092861-e777-4f57-b8b1-d69b5e0a73ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:26.521370 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8ac0a-40d8-4d25-ab89-44dd6e0cb689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:26.522139 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8ac0a-40d8-4d25-ab89-44dd6e0cb689 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:26.522507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21092861-e777-4f57-b8b1-d69b5e0a73ee None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:26.522737 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8ac0a-40d8-4d25-ab89-44dd6e0cb689 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:26.523192 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec8ac0a-40d8-4d25-ab89-44dd6e0cb689 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:26.541000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 343/679] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:26.541000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21092861-e777-4f57-b8b1-d69b5e0a73ee None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:26.588871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:26.636834 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec8ac0a-40d8-4d25-ab89-44dd6e0cb689 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:26.639249 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec8ac0a-40d8-4d25-ab89-44dd6e0cb689 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:26.683191 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21092861-e777-4f57-b8b1-d69b5e0a73ee None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:26.735234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2558e8c-d5d3-4323-8161-e7ed61dbf690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:26.735761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2558e8c-d5d3-4323-8161-e7ed61dbf690 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:26.736633 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2558e8c-d5d3-4323-8161-e7ed61dbf690 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:26.736973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2558e8c-d5d3-4323-8161-e7ed61dbf690 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:26.737691 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2558e8c-d5d3-4323-8161-e7ed61dbf690 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:26.738064 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2558e8c-d5d3-4323-8161-e7ed61dbf690 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:26.739845 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2558e8c-d5d3-4323-8161-e7ed61dbf690 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:26.742005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:26.754945 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 344/680] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:26.755588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2558e8c-d5d3-4323-8161-e7ed61dbf690 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:26.784459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 337/681] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:26.784459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8ac0a-40d8-4d25-ab89-44dd6e0cb689 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:26.911548 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:26.920528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2558e8c-d5d3-4323-8161-e7ed61dbf690 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:26.933457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec8ac0a-40d8-4d25-ab89-44dd6e0cb689 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:26.964746 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced118d-38e5-473f-a4f8-762a1bebbc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:26.965659 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced118d-38e5-473f-a4f8-762a1bebbc2e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:26.966129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced118d-38e5-473f-a4f8-762a1bebbc2e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:26.966451 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced118d-38e5-473f-a4f8-762a1bebbc2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:26.967205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced118d-38e5-473f-a4f8-762a1bebbc2e None admin] RBAC: Adding query filter params (role.id=44e187abae1d4b248fd145ad9cb97ed2, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=ca225b99584e40f4a4065bf10264974d) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:26.967517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced118d-38e5-473f-a4f8-762a1bebbc2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:26.969489 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced118d-38e5-473f-a4f8-762a1bebbc2e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:26.990932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 345/682] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:26 2022] GET /identity/v3/role_assignments?user.id=ca225b99584e40f4a4065bf10264974d&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=44e187abae1d4b248fd145ad9cb97ed2 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:26.990932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced118d-38e5-473f-a4f8-762a1bebbc2e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:26.995707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9beb69b-48f0-4c2f-ab17-4f480369e830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:26.996721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9beb69b-48f0-4c2f-ab17-4f480369e830 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:26.997255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9beb69b-48f0-4c2f-ab17-4f480369e830 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:26.997724 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9beb69b-48f0-4c2f-ab17-4f480369e830 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.009097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9beb69b-48f0-4c2f-ab17-4f480369e830 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:27.011341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9beb69b-48f0-4c2f-ab17-4f480369e830 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:27.031823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-c9beb69b-48f0-4c2f-ab17-4f480369e830 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:27.031823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:27.031823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:27.031823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:27.031823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:27.031823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:27.031823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:27.031823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:27.031823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:27.031823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:27.031823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:27.031823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:27.031823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:27.031823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:27.031823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:27.033960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:27.036524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:27.036524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:27.036524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:27.036524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:27.036524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:27.036524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:27.036524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:27.036524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:27.036524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 338/683] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:27.036524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98213a7-998d-45cd-855b-de3384f53027 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.038357 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98213a7-998d-45cd-855b-de3384f53027 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.038357 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98213a7-998d-45cd-855b-de3384f53027 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.044482 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 339/684] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:27.047911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2970b6-59b2-424d-aabb-b2c07255a557 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.049354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2970b6-59b2-424d-aabb-b2c07255a557 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.049954 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2970b6-59b2-424d-aabb-b2c07255a557 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.052522 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 340/685] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:27.053625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2970b6-59b2-424d-aabb-b2c07255a557 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:27.111561 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:27.157538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced118d-38e5-473f-a4f8-762a1bebbc2e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:27.200550 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002dd73-4a2a-4d05-b6fc-13cdad9d6a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:27.201107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002dd73-4a2a-4d05-b6fc-13cdad9d6a95 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.201385 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002dd73-4a2a-4d05-b6fc-13cdad9d6a95 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.201720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002dd73-4a2a-4d05-b6fc-13cdad9d6a95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.213981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002dd73-4a2a-4d05-b6fc-13cdad9d6a95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:27.215779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002dd73-4a2a-4d05-b6fc-13cdad9d6a95 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:27.219631 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2970b6-59b2-424d-aabb-b2c07255a557 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:27.235182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-2002dd73-4a2a-4d05-b6fc-13cdad9d6a95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:27.235182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:27.235182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:27.235182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:27.235182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:27.235182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:27.235182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:27.235182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:27.235182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:27.235182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:27.235182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:27.235182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:27.235182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:27.235182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:27.235182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:27.235182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:27.236344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:27.237388 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:27.237388 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:27.237388 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:27.237388 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 346/686] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:27.237749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ff64f-a235-4c59-a31f-8cd80d97dc3d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.238008 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ff64f-a235-4c59-a31f-8cd80d97dc3d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.238283 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ff64f-a235-4c59-a31f-8cd80d97dc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.259149 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-942ff64f-a235-4c59-a31f-8cd80d97dc3d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:27.260400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-942ff64f-a235-4c59-a31f-8cd80d97dc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:27.263428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:27.274231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb52aa5-ffa8-412c-8494-5e5a1a21eb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:27.275208 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb52aa5-ffa8-412c-8494-5e5a1a21eb25 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.275490 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb52aa5-ffa8-412c-8494-5e5a1a21eb25 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.275839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb52aa5-ffa8-412c-8494-5e5a1a21eb25 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.276999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb52aa5-ffa8-412c-8494-5e5a1a21eb25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:27.277447 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb52aa5-ffa8-412c-8494-5e5a1a21eb25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:27.279083 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb52aa5-ffa8-412c-8494-5e5a1a21eb25 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:27.295366 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 341/687] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:27.296795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb941886-31e9-4642-bb66-50c62432f5ec None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.297078 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb941886-31e9-4642-bb66-50c62432f5ec None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.297351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb941886-31e9-4642-bb66-50c62432f5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.319428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-cb941886-31e9-4642-bb66-50c62432f5ec None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:27.320637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-cb941886-31e9-4642-bb66-50c62432f5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:27.376324 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 347/688] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:27.378306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ff64f-a235-4c59-a31f-8cd80d97dc3d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:27.447286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 342/689] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:27.449552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb941886-31e9-4642-bb66-50c62432f5ec None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:27.517855 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:27.571775 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ff64f-a235-4c59-a31f-8cd80d97dc3d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:27.634357 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb941886-31e9-4642-bb66-50c62432f5ec None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:27.659612 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91687a5c-0486-4d22-994b-07ab3995e3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:27.660234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91687a5c-0486-4d22-994b-07ab3995e3d2 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.660513 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91687a5c-0486-4d22-994b-07ab3995e3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.660784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91687a5c-0486-4d22-994b-07ab3995e3d2 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.661434 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91687a5c-0486-4d22-994b-07ab3995e3d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:27.661735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91687a5c-0486-4d22-994b-07ab3995e3d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:27.663154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91687a5c-0486-4d22-994b-07ab3995e3d2 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:27.677874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cca7d-f52c-49ab-bcbb-fbf3ba3fd4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:27.678483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cca7d-f52c-49ab-bcbb-fbf3ba3fd4ca None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.678761 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cca7d-f52c-49ab-bcbb-fbf3ba3fd4ca None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.679058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cca7d-f52c-49ab-bcbb-fbf3ba3fd4ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.679783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cca7d-f52c-49ab-bcbb-fbf3ba3fd4ca None admin] RBAC: Adding query filter params (role.id=44e187abae1d4b248fd145ad9cb97ed2, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=cdbbaa64ad4844a0b2698dee82f95f23) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:27.680095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cca7d-f52c-49ab-bcbb-fbf3ba3fd4ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:27.682651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cca7d-f52c-49ab-bcbb-fbf3ba3fd4ca None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:27.683280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:27.688119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 348/690] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:27.695224 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91687a5c-0486-4d22-994b-07ab3995e3d2 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:27.700653 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 343/691] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:27 2022] GET /identity/v3/role_assignments?user.id=cdbbaa64ad4844a0b2698dee82f95f23&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=44e187abae1d4b248fd145ad9cb97ed2 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:27.732216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cca7d-f52c-49ab-bcbb-fbf3ba3fd4ca None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:27.882502 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91687a5c-0486-4d22-994b-07ab3995e3d2 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:27.907389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:27.920260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cca7d-f52c-49ab-bcbb-fbf3ba3fd4ca None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:27.928655 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd7ef0-ff03-4301-a02d-56aa6e2e3d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:27.929955 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd7ef0-ff03-4301-a02d-56aa6e2e3d6a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.934327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd7ef0-ff03-4301-a02d-56aa6e2e3d6a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.934327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd7ef0-ff03-4301-a02d-56aa6e2e3d6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.943097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd7ef0-ff03-4301-a02d-56aa6e2e3d6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:27.944724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd7ef0-ff03-4301-a02d-56aa6e2e3d6a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:27.962023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-63bd7ef0-ff03-4301-a02d-56aa6e2e3d6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:27.962023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:27.962023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:27.962023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:27.962023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:27.962023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:27.962023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:27.962023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:27.962023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:27.962023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:27.962023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:27.962023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:27.962023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:27.962023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:27.962023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:27.963111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:27.963993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:27.963993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:27.963993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:27.963993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:27.963993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:27.963993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:27.963993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:27.963993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:27.963993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 349/692] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:27.963993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd7ef0-ff03-4301-a02d-56aa6e2e3d6a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:27.973424 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f352ba-fef3-4b66-930a-f81cf9f0019a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:27.974142 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f352ba-fef3-4b66-930a-f81cf9f0019a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.975282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f352ba-fef3-4b66-930a-f81cf9f0019a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.975282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f352ba-fef3-4b66-930a-f81cf9f0019a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.990606 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f352ba-fef3-4b66-930a-f81cf9f0019a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:27.996011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-85f352ba-fef3-4b66-930a-f81cf9f0019a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:27.996011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:27.996011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:27.996011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:27.996011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:27.996011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:27.996011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:27.996011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:27.996011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:27.996011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:27.996011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:27.996011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:27.996011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:27.996011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:27.996011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:27.996011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:27.996979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:27.996979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:27.996979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:27.996979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:27.996979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:27.996979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:27.996979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:27.996979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:27.996979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:27.996979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:27.996979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:27.996979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:27.996979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:27.996979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:27.996979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:27.996979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:27.997859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:27.997859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:27.997859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:27.997859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:27.997859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:27.997859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:27.997859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:27.997859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:27.997859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:27.997859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:27.997859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:27.997859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:28.002619 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 344/693] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:28.003588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f352ba-fef3-4b66-930a-f81cf9f0019a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.049418 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.116543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd7ef0-ff03-4301-a02d-56aa6e2e3d6a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.155984 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f352ba-fef3-4b66-930a-f81cf9f0019a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.174055 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda0a75-3d28-44f0-ba08-27edfec3a844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.175123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda0a75-3d28-44f0-ba08-27edfec3a844 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.175642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda0a75-3d28-44f0-ba08-27edfec3a844 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.176425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda0a75-3d28-44f0-ba08-27edfec3a844 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.193781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda0a75-3d28-44f0-ba08-27edfec3a844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:28.196492 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cda0a75-3d28-44f0-ba08-27edfec3a844 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:28.201035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171f116-b727-4787-a972-60b982f5ae8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.201870 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171f116-b727-4787-a972-60b982f5ae8a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.202242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171f116-b727-4787-a972-60b982f5ae8a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.202646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171f116-b727-4787-a972-60b982f5ae8a None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.203472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171f116-b727-4787-a972-60b982f5ae8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:28.203911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171f116-b727-4787-a972-60b982f5ae8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:28.205627 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171f116-b727-4787-a972-60b982f5ae8a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:28.214697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-7cda0a75-3d28-44f0-ba08-27edfec3a844 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:28.214697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:28.214697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:28.214697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:28.214697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:28.214697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:28.214697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:28.214697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:28.214697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:28.214697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:28.214697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:28.214697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:28.214697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:28.214697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:28.214697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:28.214697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:28.216498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:28.218190 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:28.218190 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:28.218190 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:28.218190 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 350/694] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:28.218190 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda0a75-3d28-44f0-ba08-27edfec3a844 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.222931 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 345/695] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:28.228481 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171f116-b727-4787-a972-60b982f5ae8a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.251582 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.340540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cda0a75-3d28-44f0-ba08-27edfec3a844 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.355662 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.378049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23361129-9608-460d-b876-a976fbc99b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.378658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23361129-9608-460d-b876-a976fbc99b60 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.378966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23361129-9608-460d-b876-a976fbc99b60 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.379237 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23361129-9608-460d-b876-a976fbc99b60 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.379896 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23361129-9608-460d-b876-a976fbc99b60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:28.380286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23361129-9608-460d-b876-a976fbc99b60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:28.381780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23361129-9608-460d-b876-a976fbc99b60 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:28.385809 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171f116-b727-4787-a972-60b982f5ae8a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.391933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 351/696] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:28.393330 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23361129-9608-460d-b876-a976fbc99b60 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.439969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fb8e7-51e5-4f13-bd63-727bf6439cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.440827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fb8e7-51e5-4f13-bd63-727bf6439cd6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.441271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fb8e7-51e5-4f13-bd63-727bf6439cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.441603 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fb8e7-51e5-4f13-bd63-727bf6439cd6 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.442600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46fb8e7-51e5-4f13-bd63-727bf6439cd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:28.443130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46fb8e7-51e5-4f13-bd63-727bf6439cd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:28.445293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46fb8e7-51e5-4f13-bd63-727bf6439cd6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:28.464354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 346/697] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:28.465706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fb8e7-51e5-4f13-bd63-727bf6439cd6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.527068 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23361129-9608-460d-b876-a976fbc99b60 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.574588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.578576 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9b156-6876-4e3b-92d5-0c048fe0cbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.579640 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9b156-6876-4e3b-92d5-0c048fe0cbb1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.580742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9b156-6876-4e3b-92d5-0c048fe0cbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.581455 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9b156-6876-4e3b-92d5-0c048fe0cbb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.588183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9b156-6876-4e3b-92d5-0c048fe0cbb1 None admin] RBAC: Adding query filter params (role.id=44e187abae1d4b248fd145ad9cb97ed2, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=b3b5ae758fe847adaae0edaa0be095c9) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:28.588183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9b156-6876-4e3b-92d5-0c048fe0cbb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:28.588183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9b156-6876-4e3b-92d5-0c048fe0cbb1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:28.601506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 352/698] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/role_assignments?user.id=b3b5ae758fe847adaae0edaa0be095c9&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=44e187abae1d4b248fd145ad9cb97ed2 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:28.601506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9b156-6876-4e3b-92d5-0c048fe0cbb1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.627482 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fb8e7-51e5-4f13-bd63-727bf6439cd6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.642570 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.700317 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b9819-8469-4eb6-949e-99d4813c0520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.701227 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b9819-8469-4eb6-949e-99d4813c0520 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.701670 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b9819-8469-4eb6-949e-99d4813c0520 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.702120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b9819-8469-4eb6-949e-99d4813c0520 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.718715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9b156-6876-4e3b-92d5-0c048fe0cbb1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.722765 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b9819-8469-4eb6-949e-99d4813c0520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:28.723427 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b9819-8469-4eb6-949e-99d4813c0520 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:28.755844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-2a0b9819-8469-4eb6-949e-99d4813c0520 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:28.755844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:28.755844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:28.755844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:28.755844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:28.755844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:28.755844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:28.755844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:28.755844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:28.755844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:28.755844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:28.755844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:28.755844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:28.755844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:28.755844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:28.755844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:28.757024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:28.757922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:28.757922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:28.757922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:28.757922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 347/699] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:28.763583 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b9819-8469-4eb6-949e-99d4813c0520 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.779363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d3066-3283-44fe-928b-5209b3347d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.780331 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d3066-3283-44fe-928b-5209b3347d56 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.780795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d3066-3283-44fe-928b-5209b3347d56 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.781251 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d3066-3283-44fe-928b-5209b3347d56 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.782079 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d3066-3283-44fe-928b-5209b3347d56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:28.784394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d3066-3283-44fe-928b-5209b3347d56 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:28.894387 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 353/700] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:28 2022] POST /identity/v3/users => generated 254 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:28.946961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b9819-8469-4eb6-949e-99d4813c0520 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.991415 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.005237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8048f8-7401-43f5-9e0d-6a6a103252f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.005885 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8048f8-7401-43f5-9e0d-6a6a103252f5 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.010575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8048f8-7401-43f5-9e0d-6a6a103252f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.010575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8048f8-7401-43f5-9e0d-6a6a103252f5 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.010575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8048f8-7401-43f5-9e0d-6a6a103252f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:29.010575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8048f8-7401-43f5-9e0d-6a6a103252f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.010575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8048f8-7401-43f5-9e0d-6a6a103252f5 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.026364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 348/701] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:29.031678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d3066-3283-44fe-928b-5209b3347d56 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.195248 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.204671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d3066-3283-44fe-928b-5209b3347d56 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.250243 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d656a-47d0-460a-89b9-ea86d97bab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.252249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d656a-47d0-460a-89b9-ea86d97bab12 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.252971 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d656a-47d0-460a-89b9-ea86d97bab12 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.253638 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d656a-47d0-460a-89b9-ea86d97bab12 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.268086 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d656a-47d0-460a-89b9-ea86d97bab12 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.271431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d656a-47d0-460a-89b9-ea86d97bab12 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.289443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-d49d656a-47d0-460a-89b9-ea86d97bab12 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:29.289443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:29.289443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:29.289443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:29.289443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:29.289443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:29.289443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:29.289443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:29.289443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:29.289443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:29.289443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:29.289443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:29.289443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:29.289443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:29.289443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:29.289443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:29.291416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:29.293669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:29.293669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:29.293669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:29.293669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:29.293669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:29.293669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:29.293669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:29.293669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:29.293669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:29.293669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:29.297574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 354/702] 10.222.0.91 () {56 vars in 1093 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:29.305018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8048f8-7401-43f5-9e0d-6a6a103252f5 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.464130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8048f8-7401-43f5-9e0d-6a6a103252f5 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.521056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee11450-27d7-4c23-ad83-56787ca80d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.521572 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee11450-27d7-4c23-ad83-56787ca80d71 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.521858 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee11450-27d7-4c23-ad83-56787ca80d71 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.522131 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee11450-27d7-4c23-ad83-56787ca80d71 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.522865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee11450-27d7-4c23-ad83-56787ca80d71 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:29.523204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee11450-27d7-4c23-ad83-56787ca80d71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.524623 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee11450-27d7-4c23-ad83-56787ca80d71 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.555168 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d656a-47d0-460a-89b9-ea86d97bab12 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.555361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 349/703] 10.222.0.91 () {56 vars in 1133 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:29.594194 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eb932-cfd8-48fc-83ea-393e5bce8042 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.594662 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eb932-cfd8-48fc-83ea-393e5bce8042 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.594997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eb932-cfd8-48fc-83ea-393e5bce8042 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.597139 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 350/704] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:29.607846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01ee7c-1adc-4f40-a45c-af1bb4bbf2d5 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.608153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01ee7c-1adc-4f40-a45c-af1bb4bbf2d5 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.608392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01ee7c-1adc-4f40-a45c-af1bb4bbf2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.652823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-db01ee7c-1adc-4f40-a45c-af1bb4bbf2d5 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:29.654420 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-db01ee7c-1adc-4f40-a45c-af1bb4bbf2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:29.756157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.778986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d656a-47d0-460a-89b9-ea86d97bab12 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.826760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4338d-b9cf-4a60-ab19-d5f9fb112365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.828277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4338d-b9cf-4a60-ab19-d5f9fb112365 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.828898 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4338d-b9cf-4a60-ab19-d5f9fb112365 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.829414 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4338d-b9cf-4a60-ab19-d5f9fb112365 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.840072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 351/705] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:29.858014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.915397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4338d-b9cf-4a60-ab19-d5f9fb112365 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.917249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4338d-b9cf-4a60-ab19-d5f9fb112365 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.049625 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 355/706] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:30.055877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01ee7c-1adc-4f40-a45c-af1bb4bbf2d5 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.217507 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01ee7c-1adc-4f40-a45c-af1bb4bbf2d5 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.220713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4338d-b9cf-4a60-ab19-d5f9fb112365 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.261492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426643ba-3bcd-489d-a71c-af9f2d62b782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.262293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426643ba-3bcd-489d-a71c-af9f2d62b782 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.262575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426643ba-3bcd-489d-a71c-af9f2d62b782 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.262847 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426643ba-3bcd-489d-a71c-af9f2d62b782 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.281434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426643ba-3bcd-489d-a71c-af9f2d62b782 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.284113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426643ba-3bcd-489d-a71c-af9f2d62b782 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.303730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-426643ba-3bcd-489d-a71c-af9f2d62b782 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:30.303730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:30.303730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:30.303730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:30.303730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:30.303730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:30.303730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:30.303730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:30.303730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:30.303730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:30.303730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:30.303730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:30.303730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:30.303730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:30.303730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:30.303730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:30.304707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:30.305629 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:30.305629 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:30.305629 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:30.305629 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 352/707] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:30.312044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426643ba-3bcd-489d-a71c-af9f2d62b782 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.335406 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:30.362675 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:30.396977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4338d-b9cf-4a60-ab19-d5f9fb112365 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.442450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c0e11-b962-44b6-a556-4586cffd98ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.443142 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c0e11-b962-44b6-a556-4586cffd98ac None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.443423 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c0e11-b962-44b6-a556-4586cffd98ac None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.443696 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c0e11-b962-44b6-a556-4586cffd98ac None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.444318 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c0e11-b962-44b6-a556-4586cffd98ac None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.445904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c0e11-b962-44b6-a556-4586cffd98ac None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.449225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426643ba-3bcd-489d-a71c-af9f2d62b782 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-258c0e11-b962-44b6-a556-4586cffd98ac None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:30.459377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:30.460450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:30.460450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:30.460450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:30.460450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:30.460450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:30.460450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:30.460450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:38:30.460450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:38:30.460450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:38:30.460450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:38:30.460450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:30.460450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:30.460450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:30.460450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:30.460450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:30.460450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:30.461298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:30.461298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:30.461298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:30.461298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:30.461298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:30.461298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:30.461298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:30.461298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:30.461298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:30.461298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:30.461298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:38:30.461298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:30.461298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:38:30.461298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:30.461298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 356/708] 10.222.0.91 () {56 vars in 1129 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:30.462068 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1470b54-d68b-454c-8f7e-740f9d959b18 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.462143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1470b54-d68b-454c-8f7e-740f9d959b18 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.462766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1470b54-d68b-454c-8f7e-740f9d959b18 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.468346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 357/709] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:30.469722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1470b54-d68b-454c-8f7e-740f9d959b18 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.522056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19532f9f-4d36-4b4b-8cca-d82db4c7a52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.522995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19532f9f-4d36-4b4b-8cca-d82db4c7a52c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.523400 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19532f9f-4d36-4b4b-8cca-d82db4c7a52c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.523783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19532f9f-4d36-4b4b-8cca-d82db4c7a52c None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.524648 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19532f9f-4d36-4b4b-8cca-d82db4c7a52c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:30.525265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19532f9f-4d36-4b4b-8cca-d82db4c7a52c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.527281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19532f9f-4d36-4b4b-8cca-d82db4c7a52c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.543996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 353/710] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:30.546530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5a1d0-d1ca-4eb4-9bd0-aeb271090d7c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.546702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5a1d0-d1ca-4eb4-9bd0-aeb271090d7c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.547194 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5a1d0-d1ca-4eb4-9bd0-aeb271090d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.572852 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f4c5a1d0-d1ca-4eb4-9bd0-aeb271090d7c None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:30.573971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f4c5a1d0-d1ca-4eb4-9bd0-aeb271090d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:30.628989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1470b54-d68b-454c-8f7e-740f9d959b18 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.652920 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:30.666655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 354/711] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:30.671779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5a1d0-d1ca-4eb4-9bd0-aeb271090d7c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.680927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad571e7f-2177-4dba-886d-471e5bcaf045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.683531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad571e7f-2177-4dba-886d-471e5bcaf045 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.687372 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad571e7f-2177-4dba-886d-471e5bcaf045 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.688160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad571e7f-2177-4dba-886d-471e5bcaf045 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.689281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad571e7f-2177-4dba-886d-471e5bcaf045 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:30.690289 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad571e7f-2177-4dba-886d-471e5bcaf045 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.692773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad571e7f-2177-4dba-886d-471e5bcaf045 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.702503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 358/712] 10.222.0.91 () {56 vars in 1124 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:30.709027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad571e7f-2177-4dba-886d-471e5bcaf045 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.711282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:30.850834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad571e7f-2177-4dba-886d-471e5bcaf045 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.890293 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3080e19-6cd2-4b41-b247-0d3944c0f250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.891104 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3080e19-6cd2-4b41-b247-0d3944c0f250 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.891482 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3080e19-6cd2-4b41-b247-0d3944c0f250 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.892271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3080e19-6cd2-4b41-b247-0d3944c0f250 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.893536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3080e19-6cd2-4b41-b247-0d3944c0f250 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:30.893936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3080e19-6cd2-4b41-b247-0d3944c0f250 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.897312 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3080e19-6cd2-4b41-b247-0d3944c0f250 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.903993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 359/713] 10.222.0.91 () {56 vars in 1124 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:30.975944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5a1d0-d1ca-4eb4-9bd0-aeb271090d7c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:31.037943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3080e19-6cd2-4b41-b247-0d3944c0f250 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:31.047306 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a997890-2561-4152-85ce-50b874a51af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:31.047957 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a997890-2561-4152-85ce-50b874a51af1 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:31.048242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a997890-2561-4152-85ce-50b874a51af1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:31.048505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a997890-2561-4152-85ce-50b874a51af1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:31.068201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a997890-2561-4152-85ce-50b874a51af1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:31.069713 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a997890-2561-4152-85ce-50b874a51af1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:31.092377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-4a997890-2561-4152-85ce-50b874a51af1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:31.092377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:31.092377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:31.092377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:31.092377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:31.092377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:31.092377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:31.092377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:31.092377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:31.092377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:31.092377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:31.092377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:31.092377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:31.092377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:31.092377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:31.094039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:31.095708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:31.095708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:31.095708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:31.095708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:31.095708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:31.095708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:31.095708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:31.095708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:31.095708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 355/714] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:31.103854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a997890-2561-4152-85ce-50b874a51af1 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:31.144943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:31.240759 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:31.269840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a997890-2561-4152-85ce-50b874a51af1 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:31.284866 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3080e19-6cd2-4b41-b247-0d3944c0f250 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:31.315492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f469c0-744b-457e-ac41-1483e15ba52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:31.316508 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f469c0-744b-457e-ac41-1483e15ba52e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:31.317004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f469c0-744b-457e-ac41-1483e15ba52e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:31.317484 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f469c0-744b-457e-ac41-1483e15ba52e None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:31.318676 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f469c0-744b-457e-ac41-1483e15ba52e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:31.319215 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f469c0-744b-457e-ac41-1483e15ba52e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:31.320952 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f469c0-744b-457e-ac41-1483e15ba52e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:31.340647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 356/715] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:31.346945 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f469c0-744b-457e-ac41-1483e15ba52e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:31.377857 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d3b20-7818-4ecc-8078-d8e6c959e57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:31.378416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d3b20-7818-4ecc-8078-d8e6c959e57c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:31.378695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d3b20-7818-4ecc-8078-d8e6c959e57c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:31.378998 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d3b20-7818-4ecc-8078-d8e6c959e57c None admin] PATH_INFO: `/v3/services/network` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:31.379588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d3b20-7818-4ecc-8078-d8e6c959e57c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:31.380976 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d3b20-7818-4ecc-8078-d8e6c959e57c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-0a3d3b20-7818-4ecc-8078-d8e6c959e57c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:31.401822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:31.405351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:31.405351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:31.405351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:31.405351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:31.405351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:31.405351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:31.405351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:38:31.405351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:38:31.405351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:38:31.405351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:38:31.405351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:31.405351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:31.405351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:31.405351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:31.405351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:31.405351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:31.407706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:31.407706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:31.407706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:31.407706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:31.407706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:31.407706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:31.407706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:31.407706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:31.407706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:31.407706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:31.407706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:38:31.407706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:31.407706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:38:31.407706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:31.407706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 360/716] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:31 2022] GET /identity/v3/services/network => generated 88 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:31.423801 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d3b20-7818-4ecc-8078-d8e6c959e57c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:31.561080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f469c0-744b-457e-ac41-1483e15ba52e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:31.602659 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d3b20-7818-4ecc-8078-d8e6c959e57c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:31.605795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed0233-4d2e-4f4d-8b8d-4ef861e8ccfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:31.606411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed0233-4d2e-4f4d-8b8d-4ef861e8ccfc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:31.606699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed0233-4d2e-4f4d-8b8d-4ef861e8ccfc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:31.607029 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed0233-4d2e-4f4d-8b8d-4ef861e8ccfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:31.607781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed0233-4d2e-4f4d-8b8d-4ef861e8ccfc None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=38127f41852d412ea6ad270f1618fce2) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:31.608087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed0233-4d2e-4f4d-8b8d-4ef861e8ccfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:31.609560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed0233-4d2e-4f4d-8b8d-4ef861e8ccfc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:31.614542 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:31.616234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:31.617851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 357/717] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:31 2022] GET /identity/v3/role_assignments?user.id=38127f41852d412ea6ad270f1618fce2&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:31.645391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7055121-5a6f-4193-a823-80e8391584b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:31.646093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7055121-5a6f-4193-a823-80e8391584b3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:31.646468 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7055121-5a6f-4193-a823-80e8391584b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:31.646839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7055121-5a6f-4193-a823-80e8391584b3 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:31.647718 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7055121-5a6f-4193-a823-80e8391584b3 None admin] RBAC: Adding query filter params (name=network) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:31.648134 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7055121-5a6f-4193-a823-80e8391584b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:31.649904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7055121-5a6f-4193-a823-80e8391584b3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:31.655814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 361/718] 10.222.0.91 () {56 vars in 1112 bytes} [Thu Sep 8 13:38:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:31.661739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed0233-4d2e-4f4d-8b8d-4ef861e8ccfc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:31.884795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed0233-4d2e-4f4d-8b8d-4ef861e8ccfc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:31.930795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080a56b-de88-4b4a-aeb3-e2a1174d08a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:31.931534 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080a56b-de88-4b4a-aeb3-e2a1174d08a6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:31.931837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080a56b-de88-4b4a-aeb3-e2a1174d08a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:31.932103 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080a56b-de88-4b4a-aeb3-e2a1174d08a6 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:31.932740 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080a56b-de88-4b4a-aeb3-e2a1174d08a6 None admin] RBAC: Adding query filter params (type=network) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:31.933040 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080a56b-de88-4b4a-aeb3-e2a1174d08a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:31.934779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080a56b-de88-4b4a-aeb3-e2a1174d08a6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:31.943656 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 358/719] 10.222.0.91 () {56 vars in 1112 bytes} [Thu Sep 8 13:38:31 2022] GET /identity/v3/services?type=network => generated 124 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:32.120834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd8725-ab19-4918-b35b-092c61d89892 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:32.121521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd8725-ab19-4918-b35b-092c61d89892 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:32.121668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd8725-ab19-4918-b35b-092c61d89892 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:32.124163 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 362/720] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:32.130910 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a17df-d865-41a1-af21-13ebc2b54f90 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:32.131510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a17df-d865-41a1-af21-13ebc2b54f90 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:32.132159 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a17df-d865-41a1-af21-13ebc2b54f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:32.165931 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-928a17df-d865-41a1-af21-13ebc2b54f90 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:32.167344 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-928a17df-d865-41a1-af21-13ebc2b54f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:32.184396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2027b04-d799-4931-bbad-d69a734e2e77 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:32.184922 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2027b04-d799-4931-bbad-d69a734e2e77 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:32.185361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2027b04-d799-4931-bbad-d69a734e2e77 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:32.187585 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 363/721] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:32.193865 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62507d15-619a-4b3d-805c-221d41c9d8be None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:32.194384 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62507d15-619a-4b3d-805c-221d41c9d8be None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:32.194803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62507d15-619a-4b3d-805c-221d41c9d8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:32.237492 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-62507d15-619a-4b3d-805c-221d41c9d8be None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:32.238764 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-62507d15-619a-4b3d-805c-221d41c9d8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:32.268324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:32.289287 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 359/722] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:32.345403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 364/723] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:32.719897 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a17df-d865-41a1-af21-13ebc2b54f90 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:32.723074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62507d15-619a-4b3d-805c-221d41c9d8be None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:32.924290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a17df-d865-41a1-af21-13ebc2b54f90 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:32.950642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62507d15-619a-4b3d-805c-221d41c9d8be None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:32.957095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:32.996826 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f703b9f-3b66-4e5d-8a1f-d2b5e779d1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:32.997463 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f703b9f-3b66-4e5d-8a1f-d2b5e779d1ad None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:32.997741 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f703b9f-3b66-4e5d-8a1f-d2b5e779d1ad None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:32.998009 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f703b9f-3b66-4e5d-8a1f-d2b5e779d1ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.000573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4fddd-65fd-4985-aef5-8b66ca7133ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:33.001218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4fddd-65fd-4985-aef5-8b66ca7133ab None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.001499 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4fddd-65fd-4985-aef5-8b66ca7133ab None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.001771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4fddd-65fd-4985-aef5-8b66ca7133ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.009982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f703b9f-3b66-4e5d-8a1f-d2b5e779d1ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:33.014977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-1f703b9f-3b66-4e5d-8a1f-d2b5e779d1ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:33.014977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:33.014977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:33.014977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:33.014977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:33.014977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:33.014977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:33.014977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:33.014977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:33.014977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:33.014977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:33.014977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:33.014977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:33.014977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:33.014977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:33.014977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:33.015926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:33.015926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:33.015926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:33.015926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:33.015926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:33.015926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:33.015926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:33.015926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:33.015926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:33.015926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:33.015926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:33.015926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:33.015926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:33.015926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:33.015926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:33.015926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:33.016795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:33.016795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:33.016795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:33.016795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:33.016795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:33.016795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:33.016795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:33.016795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:33.016795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:33.016795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:33.016795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:33.016795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:33.016795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 365/724] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:33.023969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f703b9f-3b66-4e5d-8a1f-d2b5e779d1ad None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:33.025474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4fddd-65fd-4985-aef5-8b66ca7133ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:33.030685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-a2a4fddd-65fd-4985-aef5-8b66ca7133ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:33.030685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:33.030685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:33.030685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:33.030685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:33.030685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:33.030685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:33.030685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:33.030685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:33.030685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:33.030685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:33.030685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:33.030685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:33.030685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:33.030685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:33.030685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:33.032991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:33.032991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:33.032991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:33.032991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:33.032991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:33.032991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:33.032991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:33.032991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:33.032991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:33.032991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:33.032991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:33.032991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:33.032991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:33.032991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:33.032991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:33.032991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:33.035567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:33.035567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:33.035567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:33.035567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:33.035567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:33.035567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:33.035567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:33.035567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:33.035567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:33.035567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:33.035567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:33.035567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:33.035567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 360/725] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:33.043961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4fddd-65fd-4985-aef5-8b66ca7133ab None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:33.131523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:33.165574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f703b9f-3b66-4e5d-8a1f-d2b5e779d1ad None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:33.182718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4fddd-65fd-4985-aef5-8b66ca7133ab None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:33.224320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7597b13-9331-409e-a1f8-9c12fe718ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:33.224961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7597b13-9331-409e-a1f8-9c12fe718ba2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.225239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7597b13-9331-409e-a1f8-9c12fe718ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.225513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7597b13-9331-409e-a1f8-9c12fe718ba2 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.226184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7597b13-9331-409e-a1f8-9c12fe718ba2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:33.226490 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7597b13-9331-409e-a1f8-9c12fe718ba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:33.227940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7597b13-9331-409e-a1f8-9c12fe718ba2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:33.243237 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631dee08-e0b8-48cb-9768-3c9c76f7dde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:33.243848 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631dee08-e0b8-48cb-9768-3c9c76f7dde3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.244123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631dee08-e0b8-48cb-9768-3c9c76f7dde3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.244396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631dee08-e0b8-48cb-9768-3c9c76f7dde3 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.245092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631dee08-e0b8-48cb-9768-3c9c76f7dde3 None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:33.245393 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631dee08-e0b8-48cb-9768-3c9c76f7dde3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:33.246778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631dee08-e0b8-48cb-9768-3c9c76f7dde3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:33.250263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 361/726] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:33.256495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7597b13-9331-409e-a1f8-9c12fe718ba2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:33.270973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 366/727] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:33.277087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631dee08-e0b8-48cb-9768-3c9c76f7dde3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:33.337659 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:33.390005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7597b13-9331-409e-a1f8-9c12fe718ba2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:33.407783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:33.435718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd8fb8-c390-4c4f-96e3-0ca121767428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:33.436283 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd8fb8-c390-4c4f-96e3-0ca121767428 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.436560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd8fb8-c390-4c4f-96e3-0ca121767428 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.436834 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd8fb8-c390-4c4f-96e3-0ca121767428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.458796 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd8fb8-c390-4c4f-96e3-0ca121767428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:33.460282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd8fb8-c390-4c4f-96e3-0ca121767428 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:33.495524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-7dbd8fb8-c390-4c4f-96e3-0ca121767428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:33.495524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:33.495524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:33.495524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:33.495524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:33.495524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:33.495524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:33.495524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:33.495524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:33.495524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:33.495524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:33.495524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:33.495524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:33.495524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:33.495524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:33.495524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:33.496433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:33.497299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:33.497299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:33.497299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:33.497299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 362/728] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:33.499901 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd8fb8-c390-4c4f-96e3-0ca121767428 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:33.512946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631dee08-e0b8-48cb-9768-3c9c76f7dde3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:33.568931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63745373-adbb-43a5-8f61-57409913acc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:33.569575 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63745373-adbb-43a5-8f61-57409913acc4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.569854 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63745373-adbb-43a5-8f61-57409913acc4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.570125 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63745373-adbb-43a5-8f61-57409913acc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.599210 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63745373-adbb-43a5-8f61-57409913acc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:33.600690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63745373-adbb-43a5-8f61-57409913acc4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:33.637157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-63745373-adbb-43a5-8f61-57409913acc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:33.637157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:33.637157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:33.637157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:33.637157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:33.637157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:33.637157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:33.637157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:33.637157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:33.637157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:33.637157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:33.637157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:33.637157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:33.637157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:33.637157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:33.637157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:33.638135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:33.639056 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:33.639056 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:33.639056 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:33.640029 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 367/729] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:33.644673 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63745373-adbb-43a5-8f61-57409913acc4 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:33.716509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd8fb8-c390-4c4f-96e3-0ca121767428 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:33.797842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:33.799597 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9243fb3-a74c-44fe-8f20-60a049414e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:33.800205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9243fb3-a74c-44fe-8f20-60a049414e6d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.800483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9243fb3-a74c-44fe-8f20-60a049414e6d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.800756 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9243fb3-a74c-44fe-8f20-60a049414e6d None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.801398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9243fb3-a74c-44fe-8f20-60a049414e6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:33.801702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9243fb3-a74c-44fe-8f20-60a049414e6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:33.803100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9243fb3-a74c-44fe-8f20-60a049414e6d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:33.827614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 363/730] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:33.834646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9243fb3-a74c-44fe-8f20-60a049414e6d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:33.845576 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63745373-adbb-43a5-8f61-57409913acc4 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:33.882414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:33.897066 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1474ed2-0379-4481-bd14-3ba4e4800676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:33.897683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1474ed2-0379-4481-bd14-3ba4e4800676 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.897962 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1474ed2-0379-4481-bd14-3ba4e4800676 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.898234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1474ed2-0379-4481-bd14-3ba4e4800676 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.899363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1474ed2-0379-4481-bd14-3ba4e4800676 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:33.899671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1474ed2-0379-4481-bd14-3ba4e4800676 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:33.901705 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1474ed2-0379-4481-bd14-3ba4e4800676 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:33.923201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 368/731] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:33.924815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce58a89-9089-4164-bc0d-594ec98f9ad4 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.925157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce58a89-9089-4164-bc0d-594ec98f9ad4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.925437 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce58a89-9089-4164-bc0d-594ec98f9ad4 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.927241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 369/732] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:33.929226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce58a89-9089-4164-bc0d-594ec98f9ad4 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:33.981169 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9243fb3-a74c-44fe-8f20-60a049414e6d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:34.043775 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43c222-8b06-477e-8fb5-a0bfc054b000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.044405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43c222-8b06-477e-8fb5-a0bfc054b000 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.044688 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43c222-8b06-477e-8fb5-a0bfc054b000 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.044960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43c222-8b06-477e-8fb5-a0bfc054b000 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.054546 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43c222-8b06-477e-8fb5-a0bfc054b000 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:34.057293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43c222-8b06-477e-8fb5-a0bfc054b000 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:34.074464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-1a43c222-8b06-477e-8fb5-a0bfc054b000 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:34.074464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:34.074464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:34.074464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:34.074464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:34.074464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:34.074464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:34.074464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:34.074464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:34.074464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:34.074464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:34.074464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:34.074464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:34.074464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:34.074464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:34.074464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:34.075744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:34.076658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:34.076658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:34.076658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:34.076658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:34.076658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:34.076658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:34.076658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:34.076658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:34.076658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:34.076658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:34.081456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 364/733] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:34.083116 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260a82f-3630-485b-946e-cb2f93df34c2 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.083596 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260a82f-3630-485b-946e-cb2f93df34c2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.084075 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260a82f-3630-485b-946e-cb2f93df34c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.104884 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce58a89-9089-4164-bc0d-594ec98f9ad4 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:34.106234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1260a82f-3630-485b-946e-cb2f93df34c2 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:34.108731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1260a82f-3630-485b-946e-cb2f93df34c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:34.169277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11e224-78f2-4825-a033-734493212d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.171272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11e224-78f2-4825-a033-734493212d76 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.172206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11e224-78f2-4825-a033-734493212d76 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.173425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11e224-78f2-4825-a033-734493212d76 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.189252 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a11e224-78f2-4825-a033-734493212d76 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:34.192817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a11e224-78f2-4825-a033-734493212d76 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:34.199031 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:34.213067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-8a11e224-78f2-4825-a033-734493212d76 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:38:34.213067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:34.213067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:34.213067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:34.213067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:34.213067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:34.213067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:34.213067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:34.213067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:34.213067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:34.213067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:34.213067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:34.213067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:34.213067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:34.213067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:34.213067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:34.219507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:34.220541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:34.220541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:34.220541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:34.220541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:34.220541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:34.220541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:34.220541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:34.220541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:34.220541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:38:34.220541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:34.220541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 370/734] 10.222.0.91 () {56 vars in 1117 bytes} [Thu Sep 8 13:38:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:34.220541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11e224-78f2-4825-a033-734493212d76 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.221194 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 365/735] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:34.221713 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260a82f-3630-485b-946e-cb2f93df34c2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.242005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:34.354143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11e224-78f2-4825-a033-734493212d76 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:34.362979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260a82f-3630-485b-946e-cb2f93df34c2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:34.415604 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aef1f2-6750-4499-8bca-9e620716f07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.416585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aef1f2-6750-4499-8bca-9e620716f07a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.417075 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aef1f2-6750-4499-8bca-9e620716f07a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.417550 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aef1f2-6750-4499-8bca-9e620716f07a None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.418476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aef1f2-6750-4499-8bca-9e620716f07a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:34.419024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aef1f2-6750-4499-8bca-9e620716f07a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:34.419561 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b7b75-8d41-4a1c-94e6-e004a00c4a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.420469 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b7b75-8d41-4a1c-94e6-e004a00c4a6a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.420758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b7b75-8d41-4a1c-94e6-e004a00c4a6a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.421046 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b7b75-8d41-4a1c-94e6-e004a00c4a6a None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.421575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aef1f2-6750-4499-8bca-9e620716f07a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:34.421782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b7b75-8d41-4a1c-94e6-e004a00c4a6a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:34.422095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b7b75-8d41-4a1c-94e6-e004a00c4a6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:34.423566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b7b75-8d41-4a1c-94e6-e004a00c4a6a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:34.447559 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 366/736] 10.222.0.91 () {56 vars in 1149 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:34.454020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aef1f2-6750-4499-8bca-9e620716f07a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.474198 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 371/737] 10.222.0.91 () {56 vars in 1135 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:34.482060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b7b75-8d41-4a1c-94e6-e004a00c4a6a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.561145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:34.601835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:34.614592 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b7b75-8d41-4a1c-94e6-e004a00c4a6a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:34.644933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aef1f2-6750-4499-8bca-9e620716f07a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:34.657586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036b82a-6b32-4a29-b3ee-fa858ed9c179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.659213 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036b82a-6b32-4a29-b3ee-fa858ed9c179 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.659693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036b82a-6b32-4a29-b3ee-fa858ed9c179 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.660464 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036b82a-6b32-4a29-b3ee-fa858ed9c179 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.701575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce38913-d075-4060-987b-0a5e9bf7e615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.702609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce38913-d075-4060-987b-0a5e9bf7e615 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.703017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce38913-d075-4060-987b-0a5e9bf7e615 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.703456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce38913-d075-4060-987b-0a5e9bf7e615 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.758691 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036b82a-6b32-4a29-b3ee-fa858ed9c179 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:34.760510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7036b82a-6b32-4a29-b3ee-fa858ed9c179 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:34.800599 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce38913-d075-4060-987b-0a5e9bf7e615 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:34.802041 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce38913-d075-4060-987b-0a5e9bf7e615 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:34.830778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:34.885828 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 372/738] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:38:34.887342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea6fb1-7f58-46d9-afcf-a927c6985692 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.887656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea6fb1-7f58-46d9-afcf-a927c6985692 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.887929 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea6fb1-7f58-46d9-afcf-a927c6985692 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.890923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 373/739] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:34.890923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea6fb1-7f58-46d9-afcf-a927c6985692 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.910239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:34.954376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 367/740] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:34.957228 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1f02f-f8af-4a84-bb02-55a537581cfb None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.957701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1f02f-f8af-4a84-bb02-55a537581cfb None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.958196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1f02f-f8af-4a84-bb02-55a537581cfb None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.961845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 368/741] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:34.961845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1f02f-f8af-4a84-bb02-55a537581cfb None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.031286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea6fb1-7f58-46d9-afcf-a927c6985692 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.071164 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb2b18-7c4b-46a4-88cd-d8ec59179927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.071912 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb2b18-7c4b-46a4-88cd-d8ec59179927 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.072217 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb2b18-7c4b-46a4-88cd-d8ec59179927 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.072494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb2b18-7c4b-46a4-88cd-d8ec59179927 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.073075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb2b18-7c4b-46a4-88cd-d8ec59179927 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.074467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb2b18-7c4b-46a4-88cd-d8ec59179927 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:35.090863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1f02f-f8af-4a84-bb02-55a537581cfb None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.129818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4fc94-9b60-4c93-842f-8f1f90157290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.130441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4fc94-9b60-4c93-842f-8f1f90157290 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.130715 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4fc94-9b60-4c93-842f-8f1f90157290 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.131022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4fc94-9b60-4c93-842f-8f1f90157290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.142121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4fc94-9b60-4c93-842f-8f1f90157290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.144182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4fc94-9b60-4c93-842f-8f1f90157290 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:35.161779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-0bd4fc94-9b60-4c93-842f-8f1f90157290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:35.161779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:35.161779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:35.161779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:35.161779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:35.161779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:35.161779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:35.161779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:35.161779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:35.161779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:35.161779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:35.161779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:35.161779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:35.161779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:35.161779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:35.161779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:35.163889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:35.165526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:35.165526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:35.165526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:35.165526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 369/742] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:35.165526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea813f49-72dc-4e0a-97b6-8719b22c9ed1 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.165526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea813f49-72dc-4e0a-97b6-8719b22c9ed1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.165526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea813f49-72dc-4e0a-97b6-8719b22c9ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.168570 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 374/743] 10.222.0.91 () {60 vars in 1141 bytes} [Thu Sep 8 13:38:34 2022] POST /identity/v3/services => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:35.176582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb2b18-7c4b-46a4-88cd-d8ec59179927 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.200829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ea813f49-72dc-4e0a-97b6-8719b22c9ed1 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:35.202845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ea813f49-72dc-4e0a-97b6-8719b22c9ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:35.232271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:35.269034 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:35.382940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 370/744] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:35 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:35.382940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc24e4-fc60-4edf-afbf-6826e5eee650 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.382940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc24e4-fc60-4edf-afbf-6826e5eee650 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.382940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc24e4-fc60-4edf-afbf-6826e5eee650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.404763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-50cc24e4-fc60-4edf-afbf-6826e5eee650 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:35.405541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb2b18-7c4b-46a4-88cd-d8ec59179927 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.405992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-50cc24e4-fc60-4edf-afbf-6826e5eee650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:35.457486 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286653ea-0ee8-4376-b1ec-1a636921d7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.458346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286653ea-0ee8-4376-b1ec-1a636921d7e3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.458624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286653ea-0ee8-4376-b1ec-1a636921d7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.458915 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286653ea-0ee8-4376-b1ec-1a636921d7e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.474513 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286653ea-0ee8-4376-b1ec-1a636921d7e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.478342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286653ea-0ee8-4376-b1ec-1a636921d7e3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:35.503814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-286653ea-0ee8-4376-b1ec-1a636921d7e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:35.503814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:35.503814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:35.503814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:35.503814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:35.503814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:35.503814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:35.503814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:35.503814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:35.503814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:35.503814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:35.503814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:35.503814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:35.503814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:35.503814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:35.503814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:35.505782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:35.507981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:35.507981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:35.507981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:35.507981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 375/745] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:35.507981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286653ea-0ee8-4376-b1ec-1a636921d7e3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.547346 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 371/746] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:35 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:35.554939 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc24e4-fc60-4edf-afbf-6826e5eee650 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.677844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:35.697872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:35.740368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc24e4-fc60-4edf-afbf-6826e5eee650 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.759989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286653ea-0ee8-4376-b1ec-1a636921d7e3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.790650 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad44348-59a3-4161-9212-24598a366f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.791594 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad44348-59a3-4161-9212-24598a366f44 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.791900 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad44348-59a3-4161-9212-24598a366f44 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.792170 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad44348-59a3-4161-9212-24598a366f44 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.792939 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad44348-59a3-4161-9212-24598a366f44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:35.793425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad44348-59a3-4161-9212-24598a366f44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.795142 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad44348-59a3-4161-9212-24598a366f44 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:35.796513 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc637ff9-a3c8-4029-8625-134838b66f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.797222 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc637ff9-a3c8-4029-8625-134838b66f7d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.797601 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc637ff9-a3c8-4029-8625-134838b66f7d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.797990 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc637ff9-a3c8-4029-8625-134838b66f7d None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.798704 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc637ff9-a3c8-4029-8625-134838b66f7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:35.799628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc637ff9-a3c8-4029-8625-134838b66f7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.801076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc637ff9-a3c8-4029-8625-134838b66f7d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:35.811423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 372/747] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:35.813157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad44348-59a3-4161-9212-24598a366f44 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.814010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 376/748] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:35.826421 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc637ff9-a3c8-4029-8625-134838b66f7d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.940720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc637ff9-a3c8-4029-8625-134838b66f7d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.971759 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:35.975380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad44348-59a3-4161-9212-24598a366f44 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.980322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd4242-b0ee-47a1-a2d5-bb0c140b5419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.980961 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd4242-b0ee-47a1-a2d5-bb0c140b5419 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.982628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd4242-b0ee-47a1-a2d5-bb0c140b5419 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.982628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd4242-b0ee-47a1-a2d5-bb0c140b5419 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.990340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd4242-b0ee-47a1-a2d5-bb0c140b5419 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.991951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd4242-b0ee-47a1-a2d5-bb0c140b5419 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.012440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-bddd4242-b0ee-47a1-a2d5-bb0c140b5419 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:36.012440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:36.012440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:36.012440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:36.012440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:36.012440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:36.012440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:36.012440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:36.012440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:36.012440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:36.012440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:36.012440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:36.012440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:36.012440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:36.012440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:36.014170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:36.016466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:36.016466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:36.016466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:36.016466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:36.016466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:36.016466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:36.016466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:36.016466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:36.016466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 377/749] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:36.016466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd4242-b0ee-47a1-a2d5-bb0c140b5419 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.031679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a55d5c-1c9b-4020-ba5c-bbe2aa6dbf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.033224 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a55d5c-1c9b-4020-ba5c-bbe2aa6dbf2d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.034119 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a55d5c-1c9b-4020-ba5c-bbe2aa6dbf2d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.034502 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a55d5c-1c9b-4020-ba5c-bbe2aa6dbf2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.048840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a55d5c-1c9b-4020-ba5c-bbe2aa6dbf2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:36.055114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-81a55d5c-1c9b-4020-ba5c-bbe2aa6dbf2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:36.055114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:36.055114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:36.055114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:36.055114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:36.055114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:36.055114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:36.055114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:36.055114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:36.055114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:36.055114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:36.055114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:36.055114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:36.055114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:36.055114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:36.055114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:36.056088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:36.056088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:36.056088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:36.056088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:36.056088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:36.056088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:36.056088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:36.056088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:36.056088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:36.056088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:36.056088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:36.056088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:36.056088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:36.056088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:36.056088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:36.056088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:36.057565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:36.057565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:36.057565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:36.057565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:36.057565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:36.057565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:36.057565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:36.057565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:36.057565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:36.057565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:36.057565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:36.057565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:36.057565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 373/750] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:36.058262 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a55d5c-1c9b-4020-ba5c-bbe2aa6dbf2d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.062926 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:36.161796 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd4242-b0ee-47a1-a2d5-bb0c140b5419 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.184019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a55d5c-1c9b-4020-ba5c-bbe2aa6dbf2d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.195977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8dab18-c63f-422c-a12f-501ac1e0dfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.196581 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dab18-c63f-422c-a12f-501ac1e0dfbd None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.196809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dab18-c63f-422c-a12f-501ac1e0dfbd None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.197078 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dab18-c63f-422c-a12f-501ac1e0dfbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.203801 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8dab18-c63f-422c-a12f-501ac1e0dfbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.205275 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8dab18-c63f-422c-a12f-501ac1e0dfbd None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.217684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-2d8dab18-c63f-422c-a12f-501ac1e0dfbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:36.217684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:36.217684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:36.217684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:36.217684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:36.217684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:36.217684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:36.217684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:36.217684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:36.217684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:36.217684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:36.217684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:36.217684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:36.217684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:36.217684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:36.218616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:36.220286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:36.220286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:36.220286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:36.220286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:36.220286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:36.220286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:36.220286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:36.220286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:36.220286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 378/751] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:36.220286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8dab18-c63f-422c-a12f-501ac1e0dfbd None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.227939 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dae8ac-62ca-489d-a6f3-ba44fb823b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.228767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dae8ac-62ca-489d-a6f3-ba44fb823b20 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.228890 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dae8ac-62ca-489d-a6f3-ba44fb823b20 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.229090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dae8ac-62ca-489d-a6f3-ba44fb823b20 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.229779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dae8ac-62ca-489d-a6f3-ba44fb823b20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:36.230082 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dae8ac-62ca-489d-a6f3-ba44fb823b20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.231548 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dae8ac-62ca-489d-a6f3-ba44fb823b20 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.244709 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 374/752] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:36.244709 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dae8ac-62ca-489d-a6f3-ba44fb823b20 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.284372 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:36.327219 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:36.339378 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8dab18-c63f-422c-a12f-501ac1e0dfbd None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.370054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dae8ac-62ca-489d-a6f3-ba44fb823b20 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.377343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eabcd8-af33-4ed3-8cdb-79cac71d2537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.378024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eabcd8-af33-4ed3-8cdb-79cac71d2537 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.378298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eabcd8-af33-4ed3-8cdb-79cac71d2537 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.378576 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eabcd8-af33-4ed3-8cdb-79cac71d2537 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.379258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eabcd8-af33-4ed3-8cdb-79cac71d2537 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.380789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eabcd8-af33-4ed3-8cdb-79cac71d2537 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.411176 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed76d0-b06c-460e-8433-5546c281ac10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.411782 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed76d0-b06c-460e-8433-5546c281ac10 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.412059 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed76d0-b06c-460e-8433-5546c281ac10 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.412333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed76d0-b06c-460e-8433-5546c281ac10 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.412991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed76d0-b06c-460e-8433-5546c281ac10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:36.413312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed76d0-b06c-460e-8433-5546c281ac10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.414704 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed76d0-b06c-460e-8433-5546c281ac10 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.423075 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 375/753] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:36.423886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed76d0-b06c-460e-8433-5546c281ac10 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.455787 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 379/754] 10.222.0.91 () {60 vars in 1141 bytes} [Thu Sep 8 13:38:36 2022] POST /identity/v3/services => generated 240 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:36.455787 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eabcd8-af33-4ed3-8cdb-79cac71d2537 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.567239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed76d0-b06c-460e-8433-5546c281ac10 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.600490 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:36.616514 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eabcd8-af33-4ed3-8cdb-79cac71d2537 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.619603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:36.621826 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d4597-f471-4d0d-be16-6db0bdb43d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.622446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d4597-f471-4d0d-be16-6db0bdb43d3e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.622725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d4597-f471-4d0d-be16-6db0bdb43d3e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.623028 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d4597-f471-4d0d-be16-6db0bdb43d3e None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.623727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d4597-f471-4d0d-be16-6db0bdb43d3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:36.624029 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d4597-f471-4d0d-be16-6db0bdb43d3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.625738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d4597-f471-4d0d-be16-6db0bdb43d3e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.647389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 376/755] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:36.648163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d4597-f471-4d0d-be16-6db0bdb43d3e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.673130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef3822-9ddd-4a14-878c-c6de832ccd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.674542 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef3822-9ddd-4a14-878c-c6de832ccd38 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.675553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef3822-9ddd-4a14-878c-c6de832ccd38 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.676179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef3822-9ddd-4a14-878c-c6de832ccd38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.677544 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef3822-9ddd-4a14-878c-c6de832ccd38 None admin] RBAC: Adding query filter params (role.id=44e187abae1d4b248fd145ad9cb97ed2, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=46cce59255af494eaa238ab8c8cddc51) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:36.678211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef3822-9ddd-4a14-878c-c6de832ccd38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.681569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ef3822-9ddd-4a14-878c-c6de832ccd38 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.694573 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 380/756] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/role_assignments?user.id=46cce59255af494eaa238ab8c8cddc51&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=44e187abae1d4b248fd145ad9cb97ed2 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:36.699047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef3822-9ddd-4a14-878c-c6de832ccd38 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.866085 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d4597-f471-4d0d-be16-6db0bdb43d3e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.891332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef3822-9ddd-4a14-878c-c6de832ccd38 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.954766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65498c9-7c5e-4182-8589-9304e976fb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.956556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65498c9-7c5e-4182-8589-9304e976fb75 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.956978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65498c9-7c5e-4182-8589-9304e976fb75 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.957390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65498c9-7c5e-4182-8589-9304e976fb75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.958272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65498c9-7c5e-4182-8589-9304e976fb75 None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=b3b5ae758fe847adaae0edaa0be095c9) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:36.958724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65498c9-7c5e-4182-8589-9304e976fb75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.961189 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65498c9-7c5e-4182-8589-9304e976fb75 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.968690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 381/757] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/role_assignments?user.id=b3b5ae758fe847adaae0edaa0be095c9&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:36.979701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50988d69-202f-4af6-8bd2-154b8632d360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.980543 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50988d69-202f-4af6-8bd2-154b8632d360 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.980924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50988d69-202f-4af6-8bd2-154b8632d360 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.981299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50988d69-202f-4af6-8bd2-154b8632d360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.000161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50988d69-202f-4af6-8bd2-154b8632d360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:37.001811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50988d69-202f-4af6-8bd2-154b8632d360 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:37.024563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-50988d69-202f-4af6-8bd2-154b8632d360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:37.024563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:37.024563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:37.024563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:37.024563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:37.024563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:37.024563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:37.024563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:37.024563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:37.024563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:37.024563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:37.024563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:37.024563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:37.024563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:37.024563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:37.024563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:37.025651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:37.026518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:37.026518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:37.026518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:37.033025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 377/758] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:37.044066 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65498c9-7c5e-4182-8589-9304e976fb75 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:37.087400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:37.165371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65498c9-7c5e-4182-8589-9304e976fb75 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:37.206807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9a3ec-3212-4b23-8d4b-509cea7d021b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:37.208223 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9a3ec-3212-4b23-8d4b-509cea7d021b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.208651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9a3ec-3212-4b23-8d4b-509cea7d021b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.209126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9a3ec-3212-4b23-8d4b-509cea7d021b None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.209921 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9a3ec-3212-4b23-8d4b-509cea7d021b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:37.210391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9a3ec-3212-4b23-8d4b-509cea7d021b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:37.211959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9a3ec-3212-4b23-8d4b-509cea7d021b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:37.224832 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 382/759] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:37.231301 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50988d69-202f-4af6-8bd2-154b8632d360 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:37.300371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:37.407193 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50988d69-202f-4af6-8bd2-154b8632d360 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:37.486946 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1db782-779b-433d-81cb-c857de9bee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:37.487586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1db782-779b-433d-81cb-c857de9bee1f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.487917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1db782-779b-433d-81cb-c857de9bee1f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.488193 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1db782-779b-433d-81cb-c857de9bee1f None admin] PATH_INFO: `/v3/users/nova` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.496341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1db782-779b-433d-81cb-c857de9bee1f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:37.497805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1db782-779b-433d-81cb-c857de9bee1f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:37.509797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-8e1db782-779b-433d-81cb-c857de9bee1f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:37.509797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:37.509797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:37.509797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:37.509797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:37.509797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:37.509797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:37.509797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:37.509797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:37.509797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:37.509797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:37.509797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:37.509797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:37.509797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:37.509797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:37.509797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:37.510988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:37.512012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:37.512012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:37.512012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:37.512012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:37.512012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:37.512012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:37.512012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:37.512012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:37.512012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:37.512012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:37.512012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 378/760] 10.222.0.91 () {56 vars in 1093 bytes} [Thu Sep 8 13:38:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:37.520647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9a3ec-3212-4b23-8d4b-509cea7d021b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:37.674642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:37.683921 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9a3ec-3212-4b23-8d4b-509cea7d021b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:37.733142 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824076f8-9784-488b-9679-b38f1711f8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:37.733675 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824076f8-9784-488b-9679-b38f1711f8e3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.733952 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824076f8-9784-488b-9679-b38f1711f8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.734223 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824076f8-9784-488b-9679-b38f1711f8e3 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.734982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824076f8-9784-488b-9679-b38f1711f8e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:37.735289 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824076f8-9784-488b-9679-b38f1711f8e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:37.736810 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824076f8-9784-488b-9679-b38f1711f8e3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:37.764060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 383/761] 10.222.0.91 () {56 vars in 1133 bytes} [Thu Sep 8 13:38:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:37.769862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1db782-779b-433d-81cb-c857de9bee1f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:37.898307 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:37.907325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1db782-779b-433d-81cb-c857de9bee1f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:37.992282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c925339-7477-47c1-8991-c3313637419c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:37.993265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c925339-7477-47c1-8991-c3313637419c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.993768 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c925339-7477-47c1-8991-c3313637419c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.994336 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c925339-7477-47c1-8991-c3313637419c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:38.021816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c925339-7477-47c1-8991-c3313637419c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:38.024361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c925339-7477-47c1-8991-c3313637419c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:38.054872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-6c925339-7477-47c1-8991-c3313637419c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:38.054872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:38.054872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:38.054872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:38.054872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:38.054872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:38.054872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:38.054872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:38.054872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:38.054872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:38.054872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:38.054872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:38.054872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:38.054872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:38.054872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:38.054872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:38.055908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:38.057200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:38.057200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:38.057200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:38.065757 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824076f8-9784-488b-9679-b38f1711f8e3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:38.066110 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 379/762] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:38.207572 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ff7d8-6242-4623-a789-9c128da4c342 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:38.208245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ff7d8-6242-4623-a789-9c128da4c342 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:38.208733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ff7d8-6242-4623-a789-9c128da4c342 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:38.210550 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 380/763] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:38.219915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45acd9-7c5a-4dca-a865-80ad219885cb None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:38.220619 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45acd9-7c5a-4dca-a865-80ad219885cb None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:38.221497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45acd9-7c5a-4dca-a865-80ad219885cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:38.262826 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-3e45acd9-7c5a-4dca-a865-80ad219885cb None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:38.264394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3e45acd9-7c5a-4dca-a865-80ad219885cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:38.287993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824076f8-9784-488b-9679-b38f1711f8e3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:38.351462 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27318148-7628-4a43-bdbb-ee810cea879b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:38.352392 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27318148-7628-4a43-bdbb-ee810cea879b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:38.352844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27318148-7628-4a43-bdbb-ee810cea879b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:38.353300 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27318148-7628-4a43-bdbb-ee810cea879b None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:38.354147 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27318148-7628-4a43-bdbb-ee810cea879b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:38.355512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27318148-7628-4a43-bdbb-ee810cea879b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:38.357092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27318148-7628-4a43-bdbb-ee810cea879b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:38.385262 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 384/764] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:38.396197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27318148-7628-4a43-bdbb-ee810cea879b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:38.396846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 381/765] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:38:38.456856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:38.754368 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27318148-7628-4a43-bdbb-ee810cea879b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:38.832758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70159ee8-9816-4c76-b063-7fe5c907add9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:38.833401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70159ee8-9816-4c76-b063-7fe5c907add9 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:38.833674 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70159ee8-9816-4c76-b063-7fe5c907add9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:38.833943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70159ee8-9816-4c76-b063-7fe5c907add9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:38.846926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70159ee8-9816-4c76-b063-7fe5c907add9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:38.846926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70159ee8-9816-4c76-b063-7fe5c907add9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:38.865246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-70159ee8-9816-4c76-b063-7fe5c907add9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:38.865246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:38.865246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:38.865246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:38.865246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:38.865246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:38.865246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:38.865246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:38.865246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:38.865246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:38.865246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:38.865246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:38.865246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:38.865246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:38.865246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:38.866310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:38.867329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:38.867329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:38.867329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:38.867329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:38.867329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:38.867329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:38.867329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:38.867329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:38.869825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 385/766] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:38.877743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45acd9-7c5a-4dca-a865-80ad219885cb None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:38.969067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70159ee8-9816-4c76-b063-7fe5c907add9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:39.015351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:39.060492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45acd9-7c5a-4dca-a865-80ad219885cb None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:39.099599 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:39.132948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70159ee8-9816-4c76-b063-7fe5c907add9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:39.147718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fd68e-4516-47e9-aa82-9d431ea1ff18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:39.148324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fd68e-4516-47e9-aa82-9d431ea1ff18 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.148603 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fd68e-4516-47e9-aa82-9d431ea1ff18 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.148956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fd68e-4516-47e9-aa82-9d431ea1ff18 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.149609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fd68e-4516-47e9-aa82-9d431ea1ff18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:39.149912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fd68e-4516-47e9-aa82-9d431ea1ff18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:39.153280 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fd68e-4516-47e9-aa82-9d431ea1ff18 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:39.179599 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 382/767] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:39.189001 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50c2c1-41ce-4916-baad-27b3340d8b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:39.189669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50c2c1-41ce-4916-baad-27b3340d8b49 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.190447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50c2c1-41ce-4916-baad-27b3340d8b49 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.190712 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50c2c1-41ce-4916-baad-27b3340d8b49 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.190836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fd68e-4516-47e9-aa82-9d431ea1ff18 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:39.191417 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50c2c1-41ce-4916-baad-27b3340d8b49 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:39.192788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50c2c1-41ce-4916-baad-27b3340d8b49 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-6a50c2c1-41ce-4916-baad-27b3340d8b49 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:39.202236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:39.203292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:39.203292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:39.203292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:39.203292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:39.203292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:39.203292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:39.203292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:38:39.203292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:38:39.203292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:38:39.203292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:38:39.203292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:39.203292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:39.203292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:39.203292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:39.203292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:39.203292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:39.204123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:39.204123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:39.204123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:39.204123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:39.204123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:39.204123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:39.204123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:39.204123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:39.204123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:39.204123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:39.204123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:38:39.204123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:39.204123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:38:39.204123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:39.204123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 386/768] 10.222.0.91 () {56 vars in 1129 bytes} [Thu Sep 8 13:38:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:39.211931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50c2c1-41ce-4916-baad-27b3340d8b49 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:39.354936 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fd68e-4516-47e9-aa82-9d431ea1ff18 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:39.389757 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe505b-cc77-496c-a7c8-66ce6138af87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:39.390444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe505b-cc77-496c-a7c8-66ce6138af87 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.390717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe505b-cc77-496c-a7c8-66ce6138af87 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.391012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe505b-cc77-496c-a7c8-66ce6138af87 None admin] PATH_INFO: `/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/38127f41852d412ea6ad270f1618fce2/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.393414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe505b-cc77-496c-a7c8-66ce6138af87 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb2bd53d40841eeb1653bd5927fb189, user_id=38127f41852d412ea6ad270f1618fce2, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:39.394838 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe505b-cc77-496c-a7c8-66ce6138af87 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:39.397228 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:39.401269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50c2c1-41ce-4916-baad-27b3340d8b49 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:39.445030 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2535fc-83be-4df2-a14f-78b6d90e97b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:39.445634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2535fc-83be-4df2-a14f-78b6d90e97b6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.445910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2535fc-83be-4df2-a14f-78b6d90e97b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.446176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2535fc-83be-4df2-a14f-78b6d90e97b6 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.446792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2535fc-83be-4df2-a14f-78b6d90e97b6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:39.447115 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2535fc-83be-4df2-a14f-78b6d90e97b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:39.448540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2535fc-83be-4df2-a14f-78b6d90e97b6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:39.449696 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:39.454625 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 387/769] 10.222.0.91 () {56 vars in 1124 bytes} [Thu Sep 8 13:38:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:39.462770 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a5602-536f-41bb-814a-3727d975457b None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.462770 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a5602-536f-41bb-814a-3727d975457b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.462770 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a5602-536f-41bb-814a-3727d975457b None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.462770 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 388/770] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:39.470622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a5602-536f-41bb-814a-3727d975457b None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:39.486808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 383/771] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:39 2022] PUT /identity/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/38127f41852d412ea6ad270f1618fce2/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:39.495620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc96c-91c1-42db-9986-77f7fd1208e3 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.497044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc96c-91c1-42db-9986-77f7fd1208e3 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.497044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc96c-91c1-42db-9986-77f7fd1208e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.548210 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c68bc96c-91c1-42db-9986-77f7fd1208e3 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:39.549593 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c68bc96c-91c1-42db-9986-77f7fd1208e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:39.646493 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a5602-536f-41bb-814a-3727d975457b None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:39.686063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 384/772] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:38:39.688113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d58f3-61b9-44a3-a88f-246f5d96866b None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.688113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d58f3-61b9-44a3-a88f-246f5d96866b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.688113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d58f3-61b9-44a3-a88f-246f5d96866b None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.690409 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 385/773] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:39.696098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cdc00-5d82-4d71-a3a9-e27fb01c2bf5 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.696376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cdc00-5d82-4d71-a3a9-e27fb01c2bf5 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.696678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219cdc00-5d82-4d71-a3a9-e27fb01c2bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.706441 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b0924f-7769-40cc-9a9a-9bc816b1f522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:39.707341 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0924f-7769-40cc-9a9a-9bc816b1f522 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.707657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0924f-7769-40cc-9a9a-9bc816b1f522 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.707929 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b0924f-7769-40cc-9a9a-9bc816b1f522 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.708565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0924f-7769-40cc-9a9a-9bc816b1f522 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:39.708869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0924f-7769-40cc-9a9a-9bc816b1f522 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:39.710280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b0924f-7769-40cc-9a9a-9bc816b1f522 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:39.717741 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 389/774] 10.222.0.91 () {56 vars in 1124 bytes} [Thu Sep 8 13:38:39 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:39.718821 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8de66a-8300-4bed-994a-94b077c2b708 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.720123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8de66a-8300-4bed-994a-94b077c2b708 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.720123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8de66a-8300-4bed-994a-94b077c2b708 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.722535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 390/775] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:39.722810 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-219cdc00-5d82-4d71-a3a9-e27fb01c2bf5 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:39.726035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-219cdc00-5d82-4d71-a3a9-e27fb01c2bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:39.728521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8de66a-8300-4bed-994a-94b077c2b708 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:39.804822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:39.827218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 386/776] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:38:39.828519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d8631-8050-498a-aec5-07604f162d6d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.828798 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d8631-8050-498a-aec5-07604f162d6d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.829072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d8631-8050-498a-aec5-07604f162d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.868812 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8de66a-8300-4bed-994a-94b077c2b708 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:39.869462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-eb2d8631-8050-498a-aec5-07604f162d6d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:39.871024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-eb2d8631-8050-498a-aec5-07604f162d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:39.921433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a457276-4b59-4b7e-b50b-f40af13e9ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:39.923216 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a457276-4b59-4b7e-b50b-f40af13e9ff9 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.923503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a457276-4b59-4b7e-b50b-f40af13e9ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.924683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a457276-4b59-4b7e-b50b-f40af13e9ff9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.925381 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a457276-4b59-4b7e-b50b-f40af13e9ff9 None admin] RBAC: Adding query filter params (interface=public, service_id=9dcdeec2d19648d8993ce09d85b57911, region_id=RegionOne) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:39.925691 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a457276-4b59-4b7e-b50b-f40af13e9ff9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:39.930332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a457276-4b59-4b7e-b50b-f40af13e9ff9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:39.951625 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 391/777] 10.222.0.91 () {56 vars in 1251 bytes} [Thu Sep 8 13:38:39 2022] GET /identity/v3/endpoints?service_id=9dcdeec2d19648d8993ce09d85b57911&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:39.957837 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a457276-4b59-4b7e-b50b-f40af13e9ff9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:39.974489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.005900 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 387/778] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:38:40.006767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d8631-8050-498a-aec5-07604f162d6d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.095291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a457276-4b59-4b7e-b50b-f40af13e9ff9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.129684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.141247 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016389ad-f98a-4062-9587-69de5a674a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.142145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016389ad-f98a-4062-9587-69de5a674a26 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.142721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016389ad-f98a-4062-9587-69de5a674a26 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.143234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016389ad-f98a-4062-9587-69de5a674a26 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.144443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016389ad-f98a-4062-9587-69de5a674a26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.145866 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016389ad-f98a-4062-9587-69de5a674a26 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.151912 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 392/779] 10.222.0.91 () {56 vars in 1087 bytes} [Thu Sep 8 13:38:39 2022] GET /identity/v3/services => generated 772 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:40.152605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016389ad-f98a-4062-9587-69de5a674a26 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.205355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d8631-8050-498a-aec5-07604f162d6d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.309606 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128598fc-8607-4b0e-ae2b-1e7b87275b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.310258 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128598fc-8607-4b0e-ae2b-1e7b87275b50 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.310539 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128598fc-8607-4b0e-ae2b-1e7b87275b50 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.310857 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128598fc-8607-4b0e-ae2b-1e7b87275b50 None admin] PATH_INFO: `/v3/services/network` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.311475 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128598fc-8607-4b0e-ae2b-1e7b87275b50 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.312925 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128598fc-8607-4b0e-ae2b-1e7b87275b50 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-128598fc-8607-4b0e-ae2b-1e7b87275b50 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:40.328997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:40.330066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:40.330066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:40.330066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:40.330066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:40.330066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:40.330066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:40.330066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:38:40.330066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:38:40.330066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:38:40.330066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:38:40.330066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:40.330066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:40.330066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:40.330066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:40.330066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:40.330066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:40.332393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:40.332393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:40.332393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:40.332393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:40.332393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:40.332393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:40.332393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:40.332393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:40.332393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:40.332393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:40.332393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:38:40.332393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:40.332393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:38:40.332393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:40.332393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 388/780] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/services/network => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:40.333263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128598fc-8607-4b0e-ae2b-1e7b87275b50 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.352651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016389ad-f98a-4062-9587-69de5a674a26 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.398774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64902fd-5919-49da-b5b6-08b32ceb56b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.400337 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64902fd-5919-49da-b5b6-08b32ceb56b6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.400610 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64902fd-5919-49da-b5b6-08b32ceb56b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.402437 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64902fd-5919-49da-b5b6-08b32ceb56b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.403248 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.419821 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64902fd-5919-49da-b5b6-08b32ceb56b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:40.425624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-a64902fd-5919-49da-b5b6-08b32ceb56b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:40.425624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:40.425624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:40.425624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:40.425624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:40.425624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:40.425624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:40.425624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:40.425624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:40.425624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:40.425624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:40.425624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:40.425624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:40.425624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:40.425624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:40.425624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:40.426578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:40.426578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:40.426578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:40.426578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:40.426578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:40.426578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:40.426578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:40.426578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:40.426578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:40.426578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:40.426578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:40.426578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:40.426578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:40.426578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:40.426578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:40.426578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:40.427647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:40.427647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:40.427647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:40.427647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:40.427647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:40.427647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:40.427647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:40.427647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:40.427647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:40.427647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:40.427647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:40.427647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:40.427647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 393/781] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:40.427647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64902fd-5919-49da-b5b6-08b32ceb56b6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.477566 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128598fc-8607-4b0e-ae2b-1e7b87275b50 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.487476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.530090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305b300-56fe-4c56-8601-0ab81c47402d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.531003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305b300-56fe-4c56-8601-0ab81c47402d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.531428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305b300-56fe-4c56-8601-0ab81c47402d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.531842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305b300-56fe-4c56-8601-0ab81c47402d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.545079 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305b300-56fe-4c56-8601-0ab81c47402d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:40.552205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-e305b300-56fe-4c56-8601-0ab81c47402d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:40.552205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:40.552205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:40.552205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:40.552205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:40.552205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:40.552205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:40.552205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:40.552205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:40.552205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:40.552205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:40.552205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:40.552205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:40.552205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:40.552205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:40.552205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:40.553375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:40.553375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:40.553375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:40.553375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:40.553375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:40.553375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:40.553375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:40.553375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:40.553375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:40.553375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:40.553375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:40.553375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:40.553375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:40.553375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:40.553375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:40.553375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:40.554393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:40.554393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:40.554393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:40.554393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:40.554393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:40.554393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:40.554393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:40.554393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:40.554393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:40.554393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:40.554393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:40.554393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:40.561805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 389/782] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:40.561805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305b300-56fe-4c56-8601-0ab81c47402d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.567595 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64902fd-5919-49da-b5b6-08b32ceb56b6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.604987 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebfe82-d95f-4ca4-ab6a-dbe6d5f2dd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.605685 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebfe82-d95f-4ca4-ab6a-dbe6d5f2dd16 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.605965 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebfe82-d95f-4ca4-ab6a-dbe6d5f2dd16 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.606283 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebfe82-d95f-4ca4-ab6a-dbe6d5f2dd16 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.607221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebfe82-d95f-4ca4-ab6a-dbe6d5f2dd16 None admin] RBAC: Adding query filter params (name=network) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:40.607539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebfe82-d95f-4ca4-ab6a-dbe6d5f2dd16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.609895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebfe82-d95f-4ca4-ab6a-dbe6d5f2dd16 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.616431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 394/783] 10.222.0.91 () {56 vars in 1112 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:40.617257 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebfe82-d95f-4ca4-ab6a-dbe6d5f2dd16 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.688190 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305b300-56fe-4c56-8601-0ab81c47402d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.699495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.729193 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93092418-5f93-4ba1-834f-a1b443cd6f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.730154 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93092418-5f93-4ba1-834f-a1b443cd6f24 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.730639 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93092418-5f93-4ba1-834f-a1b443cd6f24 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.731158 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93092418-5f93-4ba1-834f-a1b443cd6f24 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.732048 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93092418-5f93-4ba1-834f-a1b443cd6f24 None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:40.732553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93092418-5f93-4ba1-834f-a1b443cd6f24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.734784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93092418-5f93-4ba1-834f-a1b443cd6f24 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.747157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 390/784] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:40.748586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93092418-5f93-4ba1-834f-a1b443cd6f24 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.752828 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebfe82-d95f-4ca4-ab6a-dbe6d5f2dd16 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.775490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.807751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe4f12-965a-43de-889a-f31a033bbed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.808768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe4f12-965a-43de-889a-f31a033bbed3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.809282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe4f12-965a-43de-889a-f31a033bbed3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.810339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe4f12-965a-43de-889a-f31a033bbed3 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.811212 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe4f12-965a-43de-889a-f31a033bbed3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:40.811715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe4f12-965a-43de-889a-f31a033bbed3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.814272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe4f12-965a-43de-889a-f31a033bbed3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.831278 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 395/785] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:40.832573 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe4f12-965a-43de-889a-f31a033bbed3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.883828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93092418-5f93-4ba1-834f-a1b443cd6f24 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.929133 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53cfad1-ae72-41c4-b170-db798ae65673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.929738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53cfad1-ae72-41c4-b170-db798ae65673 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.930013 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53cfad1-ae72-41c4-b170-db798ae65673 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.930280 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53cfad1-ae72-41c4-b170-db798ae65673 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.931038 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53cfad1-ae72-41c4-b170-db798ae65673 None admin] RBAC: Adding query filter params (type=network) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:40.931342 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53cfad1-ae72-41c4-b170-db798ae65673 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.933111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53cfad1-ae72-41c4-b170-db798ae65673 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.938600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 391/786] 10.222.0.91 () {56 vars in 1112 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:40.939906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53cfad1-ae72-41c4-b170-db798ae65673 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.975119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe4f12-965a-43de-889a-f31a033bbed3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.981260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.026365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020b78e-3031-4515-ab5c-b1f033c3c2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.027430 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020b78e-3031-4515-ab5c-b1f033c3c2fe None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.027919 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020b78e-3031-4515-ab5c-b1f033c3c2fe None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.028433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020b78e-3031-4515-ab5c-b1f033c3c2fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.043281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020b78e-3031-4515-ab5c-b1f033c3c2fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.045869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020b78e-3031-4515-ab5c-b1f033c3c2fe None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.057562 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53cfad1-ae72-41c4-b170-db798ae65673 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-8020b78e-3031-4515-ab5c-b1f033c3c2fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:41.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:41.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:41.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:41.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:41.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:41.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:41.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:41.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:41.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:41.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:41.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:41.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:41.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:41.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:41.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:41.067904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:41.068786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:41.068786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:41.068786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:41.071316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 396/787] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:41.072690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020b78e-3031-4515-ab5c-b1f033c3c2fe None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.098638 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c9169-5424-43ef-933d-485cbbffea94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.099578 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c9169-5424-43ef-933d-485cbbffea94 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.099886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c9169-5424-43ef-933d-485cbbffea94 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.100249 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c9169-5424-43ef-933d-485cbbffea94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.112267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c9169-5424-43ef-933d-485cbbffea94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.112885 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.114995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c9169-5424-43ef-933d-485cbbffea94 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.131392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-7d6c9169-5424-43ef-933d-485cbbffea94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:41.131392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:41.131392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:41.131392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:41.131392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:41.131392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:41.131392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:41.131392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:41.131392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:41.131392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:41.131392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:41.131392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:41.131392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:41.131392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:41.131392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:41.131392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:41.139250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:41.140216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:41.140216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:41.140216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:41.140216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 392/788] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:41.140216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c9169-5424-43ef-933d-485cbbffea94 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.212373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020b78e-3031-4515-ab5c-b1f033c3c2fe None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.244285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.260584 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92397a2-b9ce-47b9-a415-e6655a3554a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.261067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c9169-5424-43ef-933d-485cbbffea94 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.261209 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92397a2-b9ce-47b9-a415-e6655a3554a3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.261468 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92397a2-b9ce-47b9-a415-e6655a3554a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.261744 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92397a2-b9ce-47b9-a415-e6655a3554a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.262456 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92397a2-b9ce-47b9-a415-e6655a3554a3 None admin] RBAC: Adding query filter params (interface=public, service_id=1bf6b5899aa841ef9e72f69e6d7944ef, region_id=RegionOne) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:41.262764 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92397a2-b9ce-47b9-a415-e6655a3554a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.264439 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92397a2-b9ce-47b9-a415-e6655a3554a3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.272557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 397/789] 10.222.0.91 () {56 vars in 1251 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/endpoints?service_id=1bf6b5899aa841ef9e72f69e6d7944ef&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:41.274189 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92397a2-b9ce-47b9-a415-e6655a3554a3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.302349 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050cff5-8724-47c3-b4ce-584863117372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.303485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050cff5-8724-47c3-b4ce-584863117372 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.303975 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050cff5-8724-47c3-b4ce-584863117372 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.304506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050cff5-8724-47c3-b4ce-584863117372 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.305433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050cff5-8724-47c3-b4ce-584863117372 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:41.305989 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050cff5-8724-47c3-b4ce-584863117372 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.307646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050cff5-8724-47c3-b4ce-584863117372 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.321769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 393/790] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:41.323117 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050cff5-8724-47c3-b4ce-584863117372 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.397435 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.411396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92397a2-b9ce-47b9-a415-e6655a3554a3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.440146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050cff5-8724-47c3-b4ce-584863117372 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.452934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a311cb-1c2c-426b-956d-9584e5038792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.453554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a311cb-1c2c-426b-956d-9584e5038792 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.453834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a311cb-1c2c-426b-956d-9584e5038792 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.454108 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a311cb-1c2c-426b-956d-9584e5038792 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.454749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a311cb-1c2c-426b-956d-9584e5038792 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:41.455083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a311cb-1c2c-426b-956d-9584e5038792 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.456915 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a311cb-1c2c-426b-956d-9584e5038792 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.479695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 398/791] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:41.480768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a311cb-1c2c-426b-956d-9584e5038792 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.488401 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626976e4-cfe0-407e-8887-1fdebc0d83a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.489340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626976e4-cfe0-407e-8887-1fdebc0d83a9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.489813 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626976e4-cfe0-407e-8887-1fdebc0d83a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.490291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626976e4-cfe0-407e-8887-1fdebc0d83a9 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.491257 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626976e4-cfe0-407e-8887-1fdebc0d83a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.492959 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626976e4-cfe0-407e-8887-1fdebc0d83a9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.502573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 394/792] 10.222.0.91 () {56 vars in 1087 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/services => generated 772 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:41.503812 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626976e4-cfe0-407e-8887-1fdebc0d83a9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.518126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.633217 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626976e4-cfe0-407e-8887-1fdebc0d83a9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.656530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a311cb-1c2c-426b-956d-9584e5038792 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.679798 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b447c71-d4ca-4ba1-a988-74af5318ece5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.680843 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b447c71-d4ca-4ba1-a988-74af5318ece5 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.681320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b447c71-d4ca-4ba1-a988-74af5318ece5 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.681785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b447c71-d4ca-4ba1-a988-74af5318ece5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.698381 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b447c71-d4ca-4ba1-a988-74af5318ece5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.704180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b447c71-d4ca-4ba1-a988-74af5318ece5 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.726065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-5b447c71-d4ca-4ba1-a988-74af5318ece5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:41.726065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:41.726065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:41.726065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:41.726065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:41.726065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:41.726065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:41.726065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:41.726065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:41.726065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:41.726065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:41.726065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:41.726065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:41.726065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:41.726065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:41.726065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:41.727143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:41.728068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:41.728068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:41.728068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:41.728068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:41.728068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:41.728068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:41.728068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:41.728068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:41.728068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:41.728068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:41.732479 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 395/793] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:41.735843 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6430f5c-e77a-4e7a-babb-0f6deed80089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.737307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6430f5c-e77a-4e7a-babb-0f6deed80089 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.738113 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6430f5c-e77a-4e7a-babb-0f6deed80089 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.738581 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6430f5c-e77a-4e7a-babb-0f6deed80089 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.738837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b447c71-d4ca-4ba1-a988-74af5318ece5 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.751877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6430f5c-e77a-4e7a-babb-0f6deed80089 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.753856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6430f5c-e77a-4e7a-babb-0f6deed80089 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.766983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-a6430f5c-e77a-4e7a-babb-0f6deed80089 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:38:41.766983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:41.766983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:41.766983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:41.766983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:41.766983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:41.766983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:41.766983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:41.766983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:41.766983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:41.766983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:41.766983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:41.766983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:41.766983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:41.766983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:41.766983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:41.768672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:41.769743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:41.769743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:41.769743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:41.769743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:41.769743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:41.769743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:41.769743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:41.769743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:41.769743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:38:41.769743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:41.772234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 399/794] 10.222.0.91 () {56 vars in 1117 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:41.778589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6430f5c-e77a-4e7a-babb-0f6deed80089 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.803494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.863556 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.921757 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b447c71-d4ca-4ba1-a988-74af5318ece5 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.922832 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6430f5c-e77a-4e7a-babb-0f6deed80089 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.958750 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71db49-afac-4c20-b831-5a776d9f3fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.959397 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71db49-afac-4c20-b831-5a776d9f3fe2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.959675 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71db49-afac-4c20-b831-5a776d9f3fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.959947 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71db49-afac-4c20-b831-5a776d9f3fe2 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.960667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71db49-afac-4c20-b831-5a776d9f3fe2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:41.960972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71db49-afac-4c20-b831-5a776d9f3fe2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.962415 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71db49-afac-4c20-b831-5a776d9f3fe2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.983442 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a970a5-0150-479f-8485-0ebf5a2193ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.984261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a970a5-0150-479f-8485-0ebf5a2193ab None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.984720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a970a5-0150-479f-8485-0ebf5a2193ab None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.985086 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a970a5-0150-479f-8485-0ebf5a2193ab None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.986072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a970a5-0150-479f-8485-0ebf5a2193ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:41.986482 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a970a5-0150-479f-8485-0ebf5a2193ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.988541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a970a5-0150-479f-8485-0ebf5a2193ab None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.020346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 400/795] 10.222.0.91 () {56 vars in 1149 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:42.022037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37deaac-6205-4f9d-a76b-f43df880a0c9 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.022574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37deaac-6205-4f9d-a76b-f43df880a0c9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.023101 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37deaac-6205-4f9d-a76b-f43df880a0c9 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.025222 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 401/796] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:42.032435 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37deaac-6205-4f9d-a76b-f43df880a0c9 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.112423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 396/797] 10.222.0.91 () {56 vars in 1135 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:42.113676 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee379f98-5128-42ee-bf52-925d025645b7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.113960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee379f98-5128-42ee-bf52-925d025645b7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.114225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee379f98-5128-42ee-bf52-925d025645b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.120233 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:42.136665 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ee379f98-5128-42ee-bf52-925d025645b7 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:42.138132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ee379f98-5128-42ee-bf52-925d025645b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:42.173801 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37deaac-6205-4f9d-a76b-f43df880a0c9 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.221581 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 397/798] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:38:42.222462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee379f98-5128-42ee-bf52-925d025645b7 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.226111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef4e13-e5bc-44cd-8d44-f593e354fe90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.227074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef4e13-e5bc-44cd-8d44-f593e354fe90 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.227615 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef4e13-e5bc-44cd-8d44-f593e354fe90 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.228142 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef4e13-e5bc-44cd-8d44-f593e354fe90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.240562 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef4e13-e5bc-44cd-8d44-f593e354fe90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.242469 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef4e13-e5bc-44cd-8d44-f593e354fe90 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.259307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-f2ef4e13-e5bc-44cd-8d44-f593e354fe90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:42.259307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:42.259307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:42.259307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:42.259307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:42.259307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:42.259307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:42.259307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:42.259307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:42.259307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:42.259307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:42.259307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:42.259307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:42.259307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:42.259307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:42.259307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:42.260325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:42.261323 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:42.261323 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:42.261323 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:42.263691 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 402/799] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:42.270431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef4e13-e5bc-44cd-8d44-f593e354fe90 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.290655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:42.342280 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee379f98-5128-42ee-bf52-925d025645b7 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.400829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9368d5-fdf6-45fd-81d5-3dfb535f32b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.402697 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9368d5-fdf6-45fd-81d5-3dfb535f32b8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.403845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9368d5-fdf6-45fd-81d5-3dfb535f32b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.404476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9368d5-fdf6-45fd-81d5-3dfb535f32b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.421072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef4e13-e5bc-44cd-8d44-f593e354fe90 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.433122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9368d5-fdf6-45fd-81d5-3dfb535f32b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.435170 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9368d5-fdf6-45fd-81d5-3dfb535f32b8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.466086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-9e9368d5-fdf6-45fd-81d5-3dfb535f32b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:42.466086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:42.466086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:42.466086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:42.466086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:42.466086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:42.466086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:42.466086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:42.466086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:42.466086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:42.466086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:42.466086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:42.466086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:42.466086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:42.466086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:42.466086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:42.467126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:42.468020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:42.468020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:42.468020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:42.474361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:42.476462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 398/800] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:42.482429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9368d5-fdf6-45fd-81d5-3dfb535f32b8 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.510378 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1c234-6679-4321-842f-c097b312ac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.511818 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1c234-6679-4321-842f-c097b312ac72 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.514716 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1c234-6679-4321-842f-c097b312ac72 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.515362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a1c234-6679-4321-842f-c097b312ac72 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.516654 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1c234-6679-4321-842f-c097b312ac72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:42.517782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1c234-6679-4321-842f-c097b312ac72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.520499 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a1c234-6679-4321-842f-c097b312ac72 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.565008 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 403/801] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:42.575948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1c234-6679-4321-842f-c097b312ac72 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.639792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9368d5-fdf6-45fd-81d5-3dfb535f32b8 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.652410 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:42.681229 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751077e-2ad5-4dd5-8b10-479c9905a11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.681901 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751077e-2ad5-4dd5-8b10-479c9905a11d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.682202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751077e-2ad5-4dd5-8b10-479c9905a11d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.682800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751077e-2ad5-4dd5-8b10-479c9905a11d None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.683478 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751077e-2ad5-4dd5-8b10-479c9905a11d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:42.683780 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751077e-2ad5-4dd5-8b10-479c9905a11d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.685153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751077e-2ad5-4dd5-8b10-479c9905a11d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.698427 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 399/802] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:42.699471 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751077e-2ad5-4dd5-8b10-479c9905a11d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.731786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a1c234-6679-4321-842f-c097b312ac72 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.772106 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9ed0a-31e7-4434-bea0-5ec4d9022278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.772666 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9ed0a-31e7-4434-bea0-5ec4d9022278 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.772940 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9ed0a-31e7-4434-bea0-5ec4d9022278 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.773206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9ed0a-31e7-4434-bea0-5ec4d9022278 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.781732 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9ed0a-31e7-4434-bea0-5ec4d9022278 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.783286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9ed0a-31e7-4434-bea0-5ec4d9022278 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.796649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-9da9ed0a-31e7-4434-bea0-5ec4d9022278 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:42.796649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:42.796649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:42.796649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:42.796649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:42.796649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:42.796649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:42.796649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:42.796649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:42.796649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:42.796649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:42.796649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:42.796649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:42.796649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:42.796649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:42.797629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:42.798530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:42.798530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:42.798530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:42.798530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:42.798530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:42.798530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:42.798530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:42.798530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:42.802618 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 404/803] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:42.804459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9ed0a-31e7-4434-bea0-5ec4d9022278 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.842310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751077e-2ad5-4dd5-8b10-479c9905a11d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.853268 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:42.889408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d239f9-b039-4ca8-b49d-34ec478032c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.890046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d239f9-b039-4ca8-b49d-34ec478032c3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.890326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d239f9-b039-4ca8-b49d-34ec478032c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.890598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d239f9-b039-4ca8-b49d-34ec478032c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.903359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d239f9-b039-4ca8-b49d-34ec478032c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:42.909054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-e4d239f9-b039-4ca8-b49d-34ec478032c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:42.909054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:42.909054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:42.909054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:42.909054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:42.909054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:42.909054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:42.909054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:42.909054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:42.909054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:42.909054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:42.909054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:42.909054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:42.909054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:42.909054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:42.909054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:42.909981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:42.909981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:42.909981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:42.909981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:42.909981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:42.909981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:42.909981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:42.909981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:42.909981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:42.909981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:42.909981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:42.909981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:42.909981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:42.909981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:42.909981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:42.909981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:42.910832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:42.910832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:42.910832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:42.910832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:42.910832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:42.910832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:42.910832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:42.910832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:42.910832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:42.910832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:42.910832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:42.910832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:42.910832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 400/804] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:42.912109 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d239f9-b039-4ca8-b49d-34ec478032c3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.930443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9ed0a-31e7-4434-bea0-5ec4d9022278 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.956691 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:42.968105 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b50e5a-eed7-4227-a976-5c952f1df569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.976477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b50e5a-eed7-4227-a976-5c952f1df569 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.976477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b50e5a-eed7-4227-a976-5c952f1df569 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.976477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b50e5a-eed7-4227-a976-5c952f1df569 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.976477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b50e5a-eed7-4227-a976-5c952f1df569 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.977496 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b50e5a-eed7-4227-a976-5c952f1df569 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.988968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-67b50e5a-eed7-4227-a976-5c952f1df569 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:42.988968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:42.988968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:42.988968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:42.988968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:42.988968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:42.988968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:42.988968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:42.988968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:42.988968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:42.988968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:42.988968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:42.988968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:42.988968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:42.988968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:42.989861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:42.990729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:42.990729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:42.990729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:42.990729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:42.990729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:42.990729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:42.990729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:42.990729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:42.990729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 405/805] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:42.991641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602dbeed-2617-4614-87d4-14a1d20f5d4d None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.991641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602dbeed-2617-4614-87d4-14a1d20f5d4d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.991978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602dbeed-2617-4614-87d4-14a1d20f5d4d None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.997557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 406/806] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:42.998363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602dbeed-2617-4614-87d4-14a1d20f5d4d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.065847 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d239f9-b039-4ca8-b49d-34ec478032c3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.106998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a02721-a430-4c3f-8835-6b960484d7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.107883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a02721-a430-4c3f-8835-6b960484d7d9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.108208 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a02721-a430-4c3f-8835-6b960484d7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.108476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a02721-a430-4c3f-8835-6b960484d7d9 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.109156 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a02721-a430-4c3f-8835-6b960484d7d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:43.109457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a02721-a430-4c3f-8835-6b960484d7d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.110867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a02721-a430-4c3f-8835-6b960484d7d9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.116863 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602dbeed-2617-4614-87d4-14a1d20f5d4d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.119014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:43.129077 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 401/807] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:43.129077 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3cdcb-4dc4-4e50-830a-21c4a538da3a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.129077 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3cdcb-4dc4-4e50-830a-21c4a538da3a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.129077 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3cdcb-4dc4-4e50-830a-21c4a538da3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.146483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-2dc3cdcb-4dc4-4e50-830a-21c4a538da3a None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:43.147703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2dc3cdcb-4dc4-4e50-830a-21c4a538da3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:43.155320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fc2f4-6021-42ac-b35f-c5f6b7da412c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.155925 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fc2f4-6021-42ac-b35f-c5f6b7da412c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.156202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fc2f4-6021-42ac-b35f-c5f6b7da412c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.156474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fc2f4-6021-42ac-b35f-c5f6b7da412c None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.157129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fc2f4-6021-42ac-b35f-c5f6b7da412c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:43.157426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fc2f4-6021-42ac-b35f-c5f6b7da412c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.158815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fc2f4-6021-42ac-b35f-c5f6b7da412c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.167598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 407/808] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:43.168404 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fc2f4-6021-42ac-b35f-c5f6b7da412c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.230983 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 402/809] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:38:43.230983 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3cdcb-4dc4-4e50-830a-21c4a538da3a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.258714 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:43.300112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fc2f4-6021-42ac-b35f-c5f6b7da412c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.339058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2efc77-eaf8-47f8-b394-4387cd48c828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.339979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2efc77-eaf8-47f8-b394-4387cd48c828 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.340339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2efc77-eaf8-47f8-b394-4387cd48c828 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.340975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2efc77-eaf8-47f8-b394-4387cd48c828 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.341585 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2efc77-eaf8-47f8-b394-4387cd48c828 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:43.342005 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2efc77-eaf8-47f8-b394-4387cd48c828 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.343886 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2efc77-eaf8-47f8-b394-4387cd48c828 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.362665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 408/810] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:43.362665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2efc77-eaf8-47f8-b394-4387cd48c828 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.375395 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3cdcb-4dc4-4e50-830a-21c4a538da3a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.388647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:43.424271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02f532-56e6-4d0e-8bcc-45d9acdadbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.425792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02f532-56e6-4d0e-8bcc-45d9acdadbd9 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.426816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02f532-56e6-4d0e-8bcc-45d9acdadbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.427206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02f532-56e6-4d0e-8bcc-45d9acdadbd9 None admin] PATH_INFO: `/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/46cce59255af494eaa238ab8c8cddc51/roles/44e187abae1d4b248fd145ad9cb97ed2` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.444862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02f532-56e6-4d0e-8bcc-45d9acdadbd9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb2bd53d40841eeb1653bd5927fb189, user_id=46cce59255af494eaa238ab8c8cddc51, role_id=44e187abae1d4b248fd145ad9cb97ed2)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.446444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02f532-56e6-4d0e-8bcc-45d9acdadbd9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.489422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2efc77-eaf8-47f8-b394-4387cd48c828 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.527321 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377fe19-f834-4b35-ac45-ddf2ba34a866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.528246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377fe19-f834-4b35-ac45-ddf2ba34a866 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.528635 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377fe19-f834-4b35-ac45-ddf2ba34a866 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.529074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377fe19-f834-4b35-ac45-ddf2ba34a866 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.542332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377fe19-f834-4b35-ac45-ddf2ba34a866 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.543803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377fe19-f834-4b35-ac45-ddf2ba34a866 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.552850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 403/811] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:43 2022] PUT /identity/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/46cce59255af494eaa238ab8c8cddc51/roles/44e187abae1d4b248fd145ad9cb97ed2 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:43.555690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02f532-56e6-4d0e-8bcc-45d9acdadbd9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.570874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-2377fe19-f834-4b35-ac45-ddf2ba34a866 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:43.570874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:43.570874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:43.570874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:43.570874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:43.570874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:43.570874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:43.570874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:43.570874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:43.570874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:43.570874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:43.570874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:43.570874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:43.570874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:43.570874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:43.570874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:43.573024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:43.574859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:43.574859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:43.574859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:43.583464 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 409/812] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:43.583464 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377fe19-f834-4b35-ac45-ddf2ba34a866 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.663468 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:43.702088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:43.708378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02f532-56e6-4d0e-8bcc-45d9acdadbd9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.731151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377fe19-f834-4b35-ac45-ddf2ba34a866 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.754303 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0862a85-bfbe-4d91-aed4-8d4d7a178b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.755413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0862a85-bfbe-4d91-aed4-8d4d7a178b5c None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.756134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0862a85-bfbe-4d91-aed4-8d4d7a178b5c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.757014 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0862a85-bfbe-4d91-aed4-8d4d7a178b5c None admin] PATH_INFO: `/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/b3b5ae758fe847adaae0edaa0be095c9/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.759214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0862a85-bfbe-4d91-aed4-8d4d7a178b5c None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb2bd53d40841eeb1653bd5927fb189, user_id=b3b5ae758fe847adaae0edaa0be095c9, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.761511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0862a85-bfbe-4d91-aed4-8d4d7a178b5c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.779334 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ae2a3-4759-418b-92bb-1d0b7aa2fb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.779334 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ae2a3-4759-418b-92bb-1d0b7aa2fb2a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.779334 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ae2a3-4759-418b-92bb-1d0b7aa2fb2a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.779667 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ae2a3-4759-418b-92bb-1d0b7aa2fb2a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.780230 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ae2a3-4759-418b-92bb-1d0b7aa2fb2a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.781613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36ae2a3-4759-418b-92bb-1d0b7aa2fb2a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-c36ae2a3-4759-418b-92bb-1d0b7aa2fb2a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:43.791219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:43.793359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:43.793359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:43.793359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:43.793359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:43.793359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:43.793359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:43.793359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:38:43.793359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:38:43.793359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:38:43.793359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:38:43.793359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:43.793359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:43.793359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:43.793359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:43.793359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:43.793359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:43.796701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:43.796701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:43.796701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:43.796701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:43.796701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:43.796701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:43.796701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:43.796701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:43.796701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:43.796701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:43.796701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:38:43.796701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:43.796701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:38:43.796701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:43.796701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 410/813] 10.222.0.91 () {56 vars in 1129 bytes} [Thu Sep 8 13:38:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:43.797719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ae2a3-4759-418b-92bb-1d0b7aa2fb2a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.932924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 404/814] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:43 2022] PUT /identity/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/b3b5ae758fe847adaae0edaa0be095c9/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:43.934656 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0862a85-bfbe-4d91-aed4-8d4d7a178b5c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.938810 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ae2a3-4759-418b-92bb-1d0b7aa2fb2a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.022506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d7e1e-a20d-4fbe-b766-fca5ef7c399d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.023159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d7e1e-a20d-4fbe-b766-fca5ef7c399d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.023432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d7e1e-a20d-4fbe-b766-fca5ef7c399d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.023753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d7e1e-a20d-4fbe-b766-fca5ef7c399d None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.024627 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d7e1e-a20d-4fbe-b766-fca5ef7c399d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:44.024935 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d7e1e-a20d-4fbe-b766-fca5ef7c399d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.026352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d7e1e-a20d-4fbe-b766-fca5ef7c399d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:44.045296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.060790 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 411/815] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:44.063195 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0fac1-1260-445f-a5be-54cfd25a6b60 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.063476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0fac1-1260-445f-a5be-54cfd25a6b60 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.063749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0fac1-1260-445f-a5be-54cfd25a6b60 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.065390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 412/816] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:44.066459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0fac1-1260-445f-a5be-54cfd25a6b60 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.120552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0862a85-bfbe-4d91-aed4-8d4d7a178b5c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.162743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.169363 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2626e4-7bad-4357-8fda-a96896507532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.169979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2626e4-7bad-4357-8fda-a96896507532 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.170326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2626e4-7bad-4357-8fda-a96896507532 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.170605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2626e4-7bad-4357-8fda-a96896507532 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.171329 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2626e4-7bad-4357-8fda-a96896507532 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:44.171635 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2626e4-7bad-4357-8fda-a96896507532 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.175656 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2626e4-7bad-4357-8fda-a96896507532 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:44.180636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 405/817] 10.222.0.91 () {56 vars in 1124 bytes} [Thu Sep 8 13:38:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:44.182352 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6a2f2-4383-47a6-9223-59d4fb1c2cbe None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.182663 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6a2f2-4383-47a6-9223-59d4fb1c2cbe None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.183030 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6a2f2-4383-47a6-9223-59d4fb1c2cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.188723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0fac1-1260-445f-a5be-54cfd25a6b60 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.202970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c5b6a2f2-4383-47a6-9223-59d4fb1c2cbe None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:44.204177 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c5b6a2f2-4383-47a6-9223-59d4fb1c2cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:44.237521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc0874-34a5-4d01-91d6-362d72145b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.238466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc0874-34a5-4d01-91d6-362d72145b0c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.238619 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc0874-34a5-4d01-91d6-362d72145b0c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.238811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc0874-34a5-4d01-91d6-362d72145b0c None admin] PATH_INFO: `/v3/users/nova` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.246907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc0874-34a5-4d01-91d6-362d72145b0c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.248425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc0874-34a5-4d01-91d6-362d72145b0c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:44.262474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-a0bc0874-34a5-4d01-91d6-362d72145b0c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:44.262474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:44.262474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:44.262474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:44.262474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:44.262474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:44.262474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:44.262474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:44.262474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:44.262474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:44.262474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:44.262474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:44.262474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:44.262474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:44.262474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:44.262474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:44.263881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:44.266221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:44.266221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:44.266221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:44.266221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:44.266221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:44.266221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:44.266221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:44.266221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:44.266221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:44.266221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:44.266221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 413/818] 10.222.0.91 () {56 vars in 1093 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:44.266221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc0874-34a5-4d01-91d6-362d72145b0c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.290576 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 406/819] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:38:44.292157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6a2f2-4383-47a6-9223-59d4fb1c2cbe None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.354417 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.433850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc0874-34a5-4d01-91d6-362d72145b0c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.492391 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6a2f2-4383-47a6-9223-59d4fb1c2cbe None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.501451 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bd62b-4e82-42c1-adac-3a1939fc6cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.502186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bd62b-4e82-42c1-adac-3a1939fc6cb8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.503023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bd62b-4e82-42c1-adac-3a1939fc6cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.503307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bd62b-4e82-42c1-adac-3a1939fc6cb8 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.503916 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225bd62b-4e82-42c1-adac-3a1939fc6cb8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:44.504205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225bd62b-4e82-42c1-adac-3a1939fc6cb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.505581 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225bd62b-4e82-42c1-adac-3a1939fc6cb8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:44.521288 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 414/820] 10.222.0.91 () {56 vars in 1124 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:44.530021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bd62b-4e82-42c1-adac-3a1939fc6cb8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.531365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.569260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02683811-6e26-488d-96ce-fb0ec2991153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.570267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02683811-6e26-488d-96ce-fb0ec2991153 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.570780 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02683811-6e26-488d-96ce-fb0ec2991153 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.574182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02683811-6e26-488d-96ce-fb0ec2991153 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.575302 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02683811-6e26-488d-96ce-fb0ec2991153 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:44.575842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02683811-6e26-488d-96ce-fb0ec2991153 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.577555 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02683811-6e26-488d-96ce-fb0ec2991153 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:44.633294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 407/821] 10.222.0.91 () {56 vars in 1133 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:44.639356 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02683811-6e26-488d-96ce-fb0ec2991153 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.663117 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bd62b-4e82-42c1-adac-3a1939fc6cb8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.692038 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.719625 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7aac8-28d5-4fd2-9d58-00d760d15e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.720875 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7aac8-28d5-4fd2-9d58-00d760d15e63 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.721887 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7aac8-28d5-4fd2-9d58-00d760d15e63 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.722358 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7aac8-28d5-4fd2-9d58-00d760d15e63 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.723637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda7aac8-28d5-4fd2-9d58-00d760d15e63 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.725811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda7aac8-28d5-4fd2-9d58-00d760d15e63 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:44.787145 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02683811-6e26-488d-96ce-fb0ec2991153 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.822042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfbc04-8fda-43d5-81d9-738019c9dfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.823049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfbc04-8fda-43d5-81d9-738019c9dfcd None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.823375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfbc04-8fda-43d5-81d9-738019c9dfcd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.823769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfbc04-8fda-43d5-81d9-738019c9dfcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.875059 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.891660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 415/822] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:38:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:38:44.892792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7aac8-28d5-4fd2-9d58-00d760d15e63 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.914796 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcfbc04-8fda-43d5-81d9-738019c9dfcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.916650 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcfbc04-8fda-43d5-81d9-738019c9dfcd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.040714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7aac8-28d5-4fd2-9d58-00d760d15e63 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.082648 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7f21c-98eb-42ab-8532-e887734add2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.083864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7f21c-98eb-42ab-8532-e887734add2e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.084368 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7f21c-98eb-42ab-8532-e887734add2e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.084863 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7f21c-98eb-42ab-8532-e887734add2e None admin] PATH_INFO: `/v3/services/network` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.085667 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7f21c-98eb-42ab-8532-e887734add2e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.087578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7f21c-98eb-42ab-8532-e887734add2e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-dec7f21c-98eb-42ab-8532-e887734add2e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:45.098551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:45.099698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:45.099698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:45.099698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:45.099698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:45.099698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:45.099698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:45.099698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:38:45.099698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:38:45.099698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:38:45.099698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:38:45.099698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:45.099698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:45.099698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:45.099698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:45.099698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:45.099698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:45.100552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:45.100552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:45.100552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:45.100552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:45.100552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:45.100552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:45.100552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:45.100552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:45.100552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:45.100552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:45.100552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:38:45.100552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:45.100552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:38:45.100552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:45.103816 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 416/823] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:45.126390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7f21c-98eb-42ab-8532-e887734add2e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.180394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:45.204835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 408/824] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:38:45.213670 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfbc04-8fda-43d5-81d9-738019c9dfcd None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.251967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7f21c-98eb-42ab-8532-e887734add2e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.295878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7363e7-3fd6-43b9-9bb3-c7a80f1354b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.297081 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7363e7-3fd6-43b9-9bb3-c7a80f1354b3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.297588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7363e7-3fd6-43b9-9bb3-c7a80f1354b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.298105 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7363e7-3fd6-43b9-9bb3-c7a80f1354b3 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.300058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7363e7-3fd6-43b9-9bb3-c7a80f1354b3 None admin] RBAC: Adding query filter params (name=network) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:45.300790 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7363e7-3fd6-43b9-9bb3-c7a80f1354b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.302496 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7363e7-3fd6-43b9-9bb3-c7a80f1354b3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.308682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 417/825] 10.222.0.91 () {56 vars in 1112 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:45.318156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7363e7-3fd6-43b9-9bb3-c7a80f1354b3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.364220 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:45.382935 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfbc04-8fda-43d5-81d9-738019c9dfcd None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.427471 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10996077-e06f-451d-866a-b745df1db6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.428167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10996077-e06f-451d-866a-b745df1db6d6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.428448 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10996077-e06f-451d-866a-b745df1db6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.428723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10996077-e06f-451d-866a-b745df1db6d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.442197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10996077-e06f-451d-866a-b745df1db6d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.443701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10996077-e06f-451d-866a-b745df1db6d6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.444092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7363e7-3fd6-43b9-9bb3-c7a80f1354b3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.447113 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:45.463667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-10996077-e06f-451d-866a-b745df1db6d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:45.463667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:45.463667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:45.463667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:45.463667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:45.463667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:45.463667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:45.463667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:45.463667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:45.463667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:45.463667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:45.463667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:45.463667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:45.463667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:45.463667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:45.463667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:45.465091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:45.466320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:45.466320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:45.466320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:45.466320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 409/826] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:45.470605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10996077-e06f-451d-866a-b745df1db6d6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.491374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11ad88-d3e5-4df0-a70a-4dd300d7d5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.492257 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11ad88-d3e5-4df0-a70a-4dd300d7d5cb None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.492743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11ad88-d3e5-4df0-a70a-4dd300d7d5cb None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.493223 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11ad88-d3e5-4df0-a70a-4dd300d7d5cb None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.494416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11ad88-d3e5-4df0-a70a-4dd300d7d5cb None admin] RBAC: Adding query filter params (type=network) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:45.494978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11ad88-d3e5-4df0-a70a-4dd300d7d5cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.497368 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11ad88-d3e5-4df0-a70a-4dd300d7d5cb None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.509802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 418/827] 10.222.0.91 () {56 vars in 1112 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:45.517295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11ad88-d3e5-4df0-a70a-4dd300d7d5cb None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.650887 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10996077-e06f-451d-866a-b745df1db6d6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.723108 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe921ce6-40a5-43f4-bad5-48f87759ff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.723868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe921ce6-40a5-43f4-bad5-48f87759ff7b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.724146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe921ce6-40a5-43f4-bad5-48f87759ff7b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.724419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe921ce6-40a5-43f4-bad5-48f87759ff7b None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.725068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe921ce6-40a5-43f4-bad5-48f87759ff7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:45.725373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe921ce6-40a5-43f4-bad5-48f87759ff7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.726758 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe921ce6-40a5-43f4-bad5-48f87759ff7b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.731138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11ad88-d3e5-4df0-a70a-4dd300d7d5cb None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.733098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:45.758103 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 410/828] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:45.761033 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe921ce6-40a5-43f4-bad5-48f87759ff7b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.816161 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392332b-8544-40e4-a4ce-a5d07c4e6204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.816782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392332b-8544-40e4-a4ce-a5d07c4e6204 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.817057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392332b-8544-40e4-a4ce-a5d07c4e6204 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.817329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392332b-8544-40e4-a4ce-a5d07c4e6204 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.817988 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392332b-8544-40e4-a4ce-a5d07c4e6204 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.819401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392332b-8544-40e4-a4ce-a5d07c4e6204 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.887246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe921ce6-40a5-43f4-bad5-48f87759ff7b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.921514 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 419/829] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:38:45 2022] POST /identity/v3/endpoints => generated 337 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:45.937832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d85f7d-b25e-4e34-99ca-efd0e43febba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.938508 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d85f7d-b25e-4e34-99ca-efd0e43febba None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.938788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d85f7d-b25e-4e34-99ca-efd0e43febba None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.939069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d85f7d-b25e-4e34-99ca-efd0e43febba None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.947726 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d85f7d-b25e-4e34-99ca-efd0e43febba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.949312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d85f7d-b25e-4e34-99ca-efd0e43febba None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.963383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-e6d85f7d-b25e-4e34-99ca-efd0e43febba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:45.963383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:45.963383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:45.963383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:45.963383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:45.963383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:45.963383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:45.963383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:45.963383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:45.963383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:45.963383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:45.963383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:45.963383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:45.963383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:45.963383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:45.965530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:45.967671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:45.967671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:45.967671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:45.967671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:45.967671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:45.967671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:45.967671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:45.967671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:45.967671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 411/830] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:45.971292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392332b-8544-40e4-a4ce-a5d07c4e6204 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:46.026373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:46.149953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392332b-8544-40e4-a4ce-a5d07c4e6204 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:46.192585 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24467b07-0f9a-4d00-b859-a38b47bc64c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:46.193433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24467b07-0f9a-4d00-b859-a38b47bc64c1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.193718 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24467b07-0f9a-4d00-b859-a38b47bc64c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.193997 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24467b07-0f9a-4d00-b859-a38b47bc64c1 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.194696 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24467b07-0f9a-4d00-b859-a38b47bc64c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:46.195060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24467b07-0f9a-4d00-b859-a38b47bc64c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:46.196505 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24467b07-0f9a-4d00-b859-a38b47bc64c1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:46.216973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 420/831] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:46.222275 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d85f7d-b25e-4e34-99ca-efd0e43febba None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:46.227013 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1e45f-42a7-452b-92a2-6cd2adf2ab24 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.227293 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1e45f-42a7-452b-92a2-6cd2adf2ab24 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.227577 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1e45f-42a7-452b-92a2-6cd2adf2ab24 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.229200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 421/832] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:46.234592 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf633eda-be82-462a-a417-7443ae2a4d8b None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.234877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf633eda-be82-462a-a417-7443ae2a4d8b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.235175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf633eda-be82-462a-a417-7443ae2a4d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.263082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-cf633eda-be82-462a-a417-7443ae2a4d8b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:46.265269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-cf633eda-be82-462a-a417-7443ae2a4d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:46.302411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:46.359399 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 422/833] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:38:46.369640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d85f7d-b25e-4e34-99ca-efd0e43febba None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:46.419510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b3c7d-67f6-41f9-a4fa-bbae68167cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:46.420170 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b3c7d-67f6-41f9-a4fa-bbae68167cd9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.420535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b3c7d-67f6-41f9-a4fa-bbae68167cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.420820 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b3c7d-67f6-41f9-a4fa-bbae68167cd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.421570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b3c7d-67f6-41f9-a4fa-bbae68167cd9 None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=38127f41852d412ea6ad270f1618fce2) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:46.421881 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b3c7d-67f6-41f9-a4fa-bbae68167cd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:46.423406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b3c7d-67f6-41f9-a4fa-bbae68167cd9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:46.431340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 412/834] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:46 2022] GET /identity/v3/role_assignments?user.id=38127f41852d412ea6ad270f1618fce2&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:46.534203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba21c6a-9479-4a36-84ba-fac6824012e6 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.534755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba21c6a-9479-4a36-84ba-fac6824012e6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.534755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba21c6a-9479-4a36-84ba-fac6824012e6 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.536436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 423/835] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:46.543541 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3eff61-4792-49b3-b83f-93081a7fc41a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.543828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3eff61-4792-49b3-b83f-93081a7fc41a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.544107 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3eff61-4792-49b3-b83f-93081a7fc41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.569670 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-fe3eff61-4792-49b3-b83f-93081a7fc41a None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:46.571524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fe3eff61-4792-49b3-b83f-93081a7fc41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:46.667342 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 413/836] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:38:46.673211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba21c6a-9479-4a36-84ba-fac6824012e6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:46.691090 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:46.804524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba21c6a-9479-4a36-84ba-fac6824012e6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:46.842065 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a1308-6786-47fe-83cd-2463e7c89224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:46.842779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a1308-6786-47fe-83cd-2463e7c89224 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.843088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a1308-6786-47fe-83cd-2463e7c89224 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.843361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a1308-6786-47fe-83cd-2463e7c89224 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.858655 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84a1308-6786-47fe-83cd-2463e7c89224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:46.865447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-f84a1308-6786-47fe-83cd-2463e7c89224 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:46.865447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:46.865447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:46.865447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:46.865447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:46.865447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:46.865447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:46.865447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:46.865447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:46.865447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:46.865447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:46.865447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:46.865447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:46.865447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:46.865447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:46.865447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:46.866609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:46.866609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:46.866609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:46.866609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:46.866609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:46.866609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:46.866609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:46.866609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:46.866609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:46.866609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:46.866609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:46.866609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:46.866609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:46.866609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:46.866609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:46.866609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:46.867701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:46.867701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:46.867701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:46.867701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:46.867701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:46.867701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:46.867701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:46.867701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:46.867701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:46.867701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:46.867701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:46.867701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:46.867701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 424/837] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:46.874844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3eff61-4792-49b3-b83f-93081a7fc41a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:46.932893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:46.964745 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a1308-6786-47fe-83cd-2463e7c89224 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:46.995576 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3eff61-4792-49b3-b83f-93081a7fc41a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:47.049786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc7570-10e3-48c3-a9f0-c76946f57d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:47.051211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc7570-10e3-48c3-a9f0-c76946f57d18 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.051862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc7570-10e3-48c3-a9f0-c76946f57d18 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.052328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc7570-10e3-48c3-a9f0-c76946f57d18 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.054224 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc7570-10e3-48c3-a9f0-c76946f57d18 None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:47.054811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc7570-10e3-48c3-a9f0-c76946f57d18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:47.057632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc7570-10e3-48c3-a9f0-c76946f57d18 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:47.068678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:47.073905 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 414/838] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:47.079959 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc7570-10e3-48c3-a9f0-c76946f57d18 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:47.105198 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a1308-6786-47fe-83cd-2463e7c89224 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:47.148947 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36bb2e-e627-4f6a-98dd-ac20b19227f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:47.149673 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36bb2e-e627-4f6a-98dd-ac20b19227f5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.149981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36bb2e-e627-4f6a-98dd-ac20b19227f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.150318 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36bb2e-e627-4f6a-98dd-ac20b19227f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.161638 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36bb2e-e627-4f6a-98dd-ac20b19227f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:47.166811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-fa36bb2e-e627-4f6a-98dd-ac20b19227f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:47.166811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:47.166811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:47.166811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:47.166811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:47.166811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:47.166811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:47.166811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:47.166811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:47.166811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:47.166811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:47.166811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:47.166811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:47.166811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:47.166811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:47.166811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:47.167780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:47.167780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:47.167780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:47.167780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:47.167780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:47.167780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:47.167780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:47.167780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:47.167780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:47.167780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:47.167780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:47.167780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:47.167780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:47.167780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:47.167780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:47.167780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:47.168624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:47.168624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:47.168624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:47.168624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:47.168624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:47.168624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:47.168624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:47.168624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:47.168624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:47.168624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:47.168624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:47.168624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:47.168624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 425/839] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:47.175767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36bb2e-e627-4f6a-98dd-ac20b19227f5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:47.202429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc7570-10e3-48c3-a9f0-c76946f57d18 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:47.211083 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:47.240739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6ed4a-6732-4f73-aba7-bbe6df22fd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:47.241413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6ed4a-6732-4f73-aba7-bbe6df22fd28 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.241507 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6ed4a-6732-4f73-aba7-bbe6df22fd28 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.241732 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6ed4a-6732-4f73-aba7-bbe6df22fd28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.255797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6ed4a-6732-4f73-aba7-bbe6df22fd28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:47.258335 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6ed4a-6732-4f73-aba7-bbe6df22fd28 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:47.275046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-e5b6ed4a-6732-4f73-aba7-bbe6df22fd28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:47.275046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:47.275046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:47.275046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:47.275046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:47.275046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:47.275046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:47.275046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:47.275046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:47.275046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:47.275046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:47.275046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:47.275046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:47.275046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:47.275046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:47.275046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:47.276204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:47.277373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:47.277373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:47.277373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:47.278650 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 415/840] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:47.283820 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6ed4a-6732-4f73-aba7-bbe6df22fd28 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:47.319269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36bb2e-e627-4f6a-98dd-ac20b19227f5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:47.361647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744494be-8a88-40e7-a987-c69cdcd9b82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:47.362442 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744494be-8a88-40e7-a987-c69cdcd9b82b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.364488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744494be-8a88-40e7-a987-c69cdcd9b82b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.364488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744494be-8a88-40e7-a987-c69cdcd9b82b None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.364488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744494be-8a88-40e7-a987-c69cdcd9b82b None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:47.364488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744494be-8a88-40e7-a987-c69cdcd9b82b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:47.366291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744494be-8a88-40e7-a987-c69cdcd9b82b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:47.368010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:47.382551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 426/841] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:47.384283 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744494be-8a88-40e7-a987-c69cdcd9b82b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:47.442784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6ed4a-6732-4f73-aba7-bbe6df22fd28 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:47.498659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edb3d5-8e5c-49fe-854c-3de68f88cfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:47.500088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edb3d5-8e5c-49fe-854c-3de68f88cfe7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.500088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edb3d5-8e5c-49fe-854c-3de68f88cfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.500088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edb3d5-8e5c-49fe-854c-3de68f88cfe7 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.500620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edb3d5-8e5c-49fe-854c-3de68f88cfe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:47.500922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edb3d5-8e5c-49fe-854c-3de68f88cfe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:47.502303 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edb3d5-8e5c-49fe-854c-3de68f88cfe7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:47.517625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 416/842] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:47.523949 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edb3d5-8e5c-49fe-854c-3de68f88cfe7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:47.545526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744494be-8a88-40e7-a987-c69cdcd9b82b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:47.547719 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:47.605216 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dc908-89c1-406c-a7e2-cdf568f846fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:47.606276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dc908-89c1-406c-a7e2-cdf568f846fb None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.606723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dc908-89c1-406c-a7e2-cdf568f846fb None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.607480 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589dc908-89c1-406c-a7e2-cdf568f846fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.631402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589dc908-89c1-406c-a7e2-cdf568f846fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:47.633882 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589dc908-89c1-406c-a7e2-cdf568f846fb None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:47.651100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-589dc908-89c1-406c-a7e2-cdf568f846fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:47.651100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:47.651100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:47.651100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:47.651100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:47.651100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:47.651100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:47.651100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:47.651100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:47.651100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:47.651100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:47.651100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:47.651100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:47.651100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:47.651100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:47.651100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:47.652587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:47.654034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:47.654034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:47.654034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:47.657270 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 427/843] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:47.658300 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dc908-89c1-406c-a7e2-cdf568f846fb None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:47.670561 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edb3d5-8e5c-49fe-854c-3de68f88cfe7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:47.710507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:47.713626 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67ab30-dcc1-40dd-8408-dd710bc322dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:47.714253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67ab30-dcc1-40dd-8408-dd710bc322dd None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.714526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67ab30-dcc1-40dd-8408-dd710bc322dd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.714791 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67ab30-dcc1-40dd-8408-dd710bc322dd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.722665 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67ab30-dcc1-40dd-8408-dd710bc322dd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:47.724863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67ab30-dcc1-40dd-8408-dd710bc322dd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:47.736184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-fd67ab30-dcc1-40dd-8408-dd710bc322dd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:38:47.736184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:47.736184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:47.736184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:47.736184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:47.736184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:47.736184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:47.736184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:47.736184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:47.736184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:47.736184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:47.736184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:47.736184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:47.736184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:47.736184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:47.736184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:47.737793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:47.739485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:47.739485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:47.739485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:47.739485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:47.739485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:47.739485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:47.739485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:47.739485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:47.739485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:38:47.739485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:47.739485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 417/844] 10.222.0.91 () {56 vars in 1117 bytes} [Thu Sep 8 13:38:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:47.739485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b69f4-7981-47bd-8b6e-7c6e48cf26a1 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.740175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b69f4-7981-47bd-8b6e-7c6e48cf26a1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.740175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b69f4-7981-47bd-8b6e-7c6e48cf26a1 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.743492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 418/845] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:47.743492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b69f4-7981-47bd-8b6e-7c6e48cf26a1 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:47.773988 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589dc908-89c1-406c-a7e2-cdf568f846fb None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:47.811953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6efed-79db-4258-9b6e-38efbbfaba60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:47.812557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6efed-79db-4258-9b6e-38efbbfaba60 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.812837 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6efed-79db-4258-9b6e-38efbbfaba60 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.813108 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6efed-79db-4258-9b6e-38efbbfaba60 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.813745 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd6efed-79db-4258-9b6e-38efbbfaba60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:47.814041 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd6efed-79db-4258-9b6e-38efbbfaba60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:47.815483 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd6efed-79db-4258-9b6e-38efbbfaba60 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:47.828915 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 428/846] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:47.830131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598bdaf-d316-4807-9bb8-2cf763e6285e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.830412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598bdaf-d316-4807-9bb8-2cf763e6285e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.831201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598bdaf-d316-4807-9bb8-2cf763e6285e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.839251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:47.850214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-6598bdaf-d316-4807-9bb8-2cf763e6285e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:47.851241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6598bdaf-d316-4807-9bb8-2cf763e6285e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:47.866566 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b69f4-7981-47bd-8b6e-7c6e48cf26a1 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:47.909525 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3827a-d6f7-46b4-89d9-787e4173b8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:47.910159 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3827a-d6f7-46b4-89d9-787e4173b8c1 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.910455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3827a-d6f7-46b4-89d9-787e4173b8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.910785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3827a-d6f7-46b4-89d9-787e4173b8c1 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.912156 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3827a-d6f7-46b4-89d9-787e4173b8c1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:47.912367 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3827a-d6f7-46b4-89d9-787e4173b8c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:47.914376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3827a-d6f7-46b4-89d9-787e4173b8c1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:47.930144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 429/847] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:38:47.932989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598bdaf-d316-4807-9bb8-2cf763e6285e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:47.934980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 419/848] 10.222.0.91 () {56 vars in 1149 bytes} [Thu Sep 8 13:38:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:47.949447 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3827a-d6f7-46b4-89d9-787e4173b8c1 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:47.985598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.053236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598bdaf-d316-4807-9bb8-2cf763e6285e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.083017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3827a-d6f7-46b4-89d9-787e4173b8c1 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.102134 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ce3ef-1b7a-4e9f-8c58-d1df4eeceda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.102917 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ce3ef-1b7a-4e9f-8c58-d1df4eeceda3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.103251 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ce3ef-1b7a-4e9f-8c58-d1df4eeceda3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.103589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ce3ef-1b7a-4e9f-8c58-d1df4eeceda3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.111336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ce3ef-1b7a-4e9f-8c58-d1df4eeceda3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:48.113458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ce3ef-1b7a-4e9f-8c58-d1df4eeceda3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:48.124586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ca1ba-6069-42de-b2c4-58d50846cf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.125116 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ca1ba-6069-42de-b2c4-58d50846cf86 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.125392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ca1ba-6069-42de-b2c4-58d50846cf86 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.125663 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ca1ba-6069-42de-b2c4-58d50846cf86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.125858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-683ce3ef-1b7a-4e9f-8c58-d1df4eeceda3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:48.125858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:48.125858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:48.125858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:48.125858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:48.125858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:48.125858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:48.125858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:48.125858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:48.125858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:48.125858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:48.125858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:48.125858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:48.125858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:48.125858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:48.125858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:48.126731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:48.127774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:48.127774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:48.127774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:48.127774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:48.127774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:48.127774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:48.127774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:48.127774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:48.127774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:48.127774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:48.127774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 430/849] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:48.129790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.135478 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ce3ef-1b7a-4e9f-8c58-d1df4eeceda3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.201922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ca1ba-6069-42de-b2c4-58d50846cf86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:48.203513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ca1ba-6069-42de-b2c4-58d50846cf86 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:48.253259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.257899 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ce3ef-1b7a-4e9f-8c58-d1df4eeceda3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.293932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78b791-8fc2-4935-bf24-4a7909d38e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.294598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78b791-8fc2-4935-bf24-4a7909d38e7b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.295541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78b791-8fc2-4935-bf24-4a7909d38e7b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.295541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78b791-8fc2-4935-bf24-4a7909d38e7b None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.296576 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78b791-8fc2-4935-bf24-4a7909d38e7b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:48.296576 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78b791-8fc2-4935-bf24-4a7909d38e7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:48.297934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78b791-8fc2-4935-bf24-4a7909d38e7b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:48.310816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 420/850] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:47 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:38:48.311651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ca1ba-6069-42de-b2c4-58d50846cf86 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.327343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 431/851] 10.222.0.91 () {56 vars in 1135 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:48.328473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78b791-8fc2-4935-bf24-4a7909d38e7b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.357639 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.433101 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ca1ba-6069-42de-b2c4-58d50846cf86 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.439255 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78b791-8fc2-4935-bf24-4a7909d38e7b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.470210 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e38900-a9ec-4dc0-bc95-9c5b7d77709b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.471130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e38900-a9ec-4dc0-bc95-9c5b7d77709b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.471333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e38900-a9ec-4dc0-bc95-9c5b7d77709b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.472342 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e38900-a9ec-4dc0-bc95-9c5b7d77709b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.473252 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e38900-a9ec-4dc0-bc95-9c5b7d77709b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:48.474707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e38900-a9ec-4dc0-bc95-9c5b7d77709b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:48.488527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a158c-1839-4be9-9be3-1d3694debdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.489155 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a158c-1839-4be9-9be3-1d3694debdd7 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.489429 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a158c-1839-4be9-9be3-1d3694debdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.489697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a158c-1839-4be9-9be3-1d3694debdd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-78e38900-a9ec-4dc0-bc95-9c5b7d77709b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:48.491035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:48.492021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:48.492021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:48.492021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:48.492021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:48.492021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:48.492021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:48.492021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:38:48.492021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:38:48.492021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:38:48.492021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:38:48.492021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:48.492021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:48.492021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:48.492021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:48.492021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:48.492021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:48.492889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:48.492889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:48.492889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:48.492889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:48.492889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:48.492889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:48.492889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:48.492889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:48.492889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:48.492889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:48.492889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:38:48.492889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:48.492889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:38:48.492889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:48.495540 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 421/852] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:48.496483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e38900-a9ec-4dc0-bc95-9c5b7d77709b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.506410 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a158c-1839-4be9-9be3-1d3694debdd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:48.508039 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a158c-1839-4be9-9be3-1d3694debdd7 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:48.524433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-434a158c-1839-4be9-9be3-1d3694debdd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:48.524433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:48.524433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:48.524433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:48.524433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:48.524433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:48.524433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:48.524433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:48.524433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:48.524433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:48.524433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:48.524433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:48.524433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:48.524433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:48.524433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:48.524433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:48.525413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:48.526539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:48.526539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:48.526539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:48.526539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 432/853] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:48.526761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a158c-1839-4be9-9be3-1d3694debdd7 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.540750 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.616379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.620193 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e38900-a9ec-4dc0-bc95-9c5b7d77709b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.645959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a158c-1839-4be9-9be3-1d3694debdd7 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.655486 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5608b-0840-4b7b-8754-25473f833072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.656493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5608b-0840-4b7b-8754-25473f833072 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.657870 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5608b-0840-4b7b-8754-25473f833072 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.658488 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5608b-0840-4b7b-8754-25473f833072 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.683856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652e4f0-5683-40cd-9831-fd0e9a8fa08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.684387 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652e4f0-5683-40cd-9831-fd0e9a8fa08e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.684661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652e4f0-5683-40cd-9831-fd0e9a8fa08e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.684929 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652e4f0-5683-40cd-9831-fd0e9a8fa08e None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.685577 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652e4f0-5683-40cd-9831-fd0e9a8fa08e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:48.685848 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652e4f0-5683-40cd-9831-fd0e9a8fa08e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:48.687246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652e4f0-5683-40cd-9831-fd0e9a8fa08e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:48.696721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 433/854] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:48.696721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652e4f0-5683-40cd-9831-fd0e9a8fa08e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.745039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5608b-0840-4b7b-8754-25473f833072 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:48.746574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5608b-0840-4b7b-8754-25473f833072 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:48.766014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.801714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652e4f0-5683-40cd-9831-fd0e9a8fa08e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.837936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25a8cd-6f5a-4975-9a22-c9e60c8aed32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.838546 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25a8cd-6f5a-4975-9a22-c9e60c8aed32 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.838827 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25a8cd-6f5a-4975-9a22-c9e60c8aed32 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.839153 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc25a8cd-6f5a-4975-9a22-c9e60c8aed32 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.839763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25a8cd-6f5a-4975-9a22-c9e60c8aed32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:48.840062 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25a8cd-6f5a-4975-9a22-c9e60c8aed32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:48.841432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc25a8cd-6f5a-4975-9a22-c9e60c8aed32 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:48.853172 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.860866 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 434/855] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:48.860866 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25a8cd-6f5a-4975-9a22-c9e60c8aed32 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.861168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 422/856] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:38:48.862983 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5608b-0840-4b7b-8754-25473f833072 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.973549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc25a8cd-6f5a-4975-9a22-c9e60c8aed32 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.977883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5608b-0840-4b7b-8754-25473f833072 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.002488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.007939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10239e-ad6d-4ff2-9558-c309f660c7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.008537 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10239e-ad6d-4ff2-9558-c309f660c7e5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.008811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10239e-ad6d-4ff2-9558-c309f660c7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.009139 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10239e-ad6d-4ff2-9558-c309f660c7e5 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.009700 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10239e-ad6d-4ff2-9558-c309f660c7e5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:49.009992 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10239e-ad6d-4ff2-9558-c309f660c7e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.011496 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10239e-ad6d-4ff2-9558-c309f660c7e5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.017097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de978cb-d1c2-4bf9-8056-79b2f832a8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.017097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de978cb-d1c2-4bf9-8056-79b2f832a8ad None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.017097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de978cb-d1c2-4bf9-8056-79b2f832a8ad None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.017097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de978cb-d1c2-4bf9-8056-79b2f832a8ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.017461 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 435/857] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:49.017461 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10239e-ad6d-4ff2-9558-c309f660c7e5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.037002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de978cb-d1c2-4bf9-8056-79b2f832a8ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.038631 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de978cb-d1c2-4bf9-8056-79b2f832a8ad None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.051511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-5de978cb-d1c2-4bf9-8056-79b2f832a8ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:49.051511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:49.051511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:49.051511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:49.051511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:49.051511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:49.051511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:49.051511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:49.051511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:49.051511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:49.051511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:49.051511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:49.051511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:49.051511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:49.051511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:49.052512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:49.053451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:49.053451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:49.053451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:49.053451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:49.053451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:49.053451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:49.053451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:49.053451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:49.053451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 423/858] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:49.053936 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb76b4-3a83-4edf-9ef1-a90489a24344 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.054533 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb76b4-3a83-4edf-9ef1-a90489a24344 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.054533 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb76b4-3a83-4edf-9ef1-a90489a24344 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.055963 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 424/859] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:49.061097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb76b4-3a83-4edf-9ef1-a90489a24344 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.133567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.138766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10239e-ad6d-4ff2-9558-c309f660c7e5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.181197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a01589-ed68-4109-bc9f-6888055a11be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.181906 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a01589-ed68-4109-bc9f-6888055a11be None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.182179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a01589-ed68-4109-bc9f-6888055a11be None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.182448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a01589-ed68-4109-bc9f-6888055a11be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.183335 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb76b4-3a83-4edf-9ef1-a90489a24344 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.194089 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a01589-ed68-4109-bc9f-6888055a11be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.195895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a01589-ed68-4109-bc9f-6888055a11be None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.215175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-e3a01589-ed68-4109-bc9f-6888055a11be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:49.215175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:49.215175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:49.215175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:49.215175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:49.215175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:49.215175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:49.215175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:49.215175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:49.215175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:49.215175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:49.215175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:49.215175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:49.215175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:49.215175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:49.215175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:49.216228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:49.217097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:49.217097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:49.217097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:49.217097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 436/860] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:49.217922 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e499a4-a640-46cd-a8e0-3393d2131bf7 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.218200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e499a4-a640-46cd-a8e0-3393d2131bf7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.218469 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e499a4-a640-46cd-a8e0-3393d2131bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.226859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ccaef-f45a-4999-9c57-e2ac4de96843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.227424 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ccaef-f45a-4999-9c57-e2ac4de96843 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.227697 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ccaef-f45a-4999-9c57-e2ac4de96843 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.227968 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ccaef-f45a-4999-9c57-e2ac4de96843 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.228659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ccaef-f45a-4999-9c57-e2ac4de96843 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:49.228960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ccaef-f45a-4999-9c57-e2ac4de96843 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.230406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ccaef-f45a-4999-9c57-e2ac4de96843 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.239067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f6e499a4-a640-46cd-a8e0-3393d2131bf7 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:49.240801 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f6e499a4-a640-46cd-a8e0-3393d2131bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:49.243584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 425/861] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:49.244264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ccaef-f45a-4999-9c57-e2ac4de96843 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.307743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.337956 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 437/862] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:38:49.339762 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e499a4-a640-46cd-a8e0-3393d2131bf7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.393908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ccaef-f45a-4999-9c57-e2ac4de96843 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.418279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.435347 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb231de4-a008-4c40-a082-416616b9d03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.436045 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb231de4-a008-4c40-a082-416616b9d03d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.436395 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb231de4-a008-4c40-a082-416616b9d03d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.436740 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb231de4-a008-4c40-a082-416616b9d03d None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.437523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb231de4-a008-4c40-a082-416616b9d03d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:49.438068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb231de4-a008-4c40-a082-416616b9d03d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.440360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb231de4-a008-4c40-a082-416616b9d03d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.455015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 426/863] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:49.467952 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb231de4-a008-4c40-a082-416616b9d03d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.485894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e499a4-a640-46cd-a8e0-3393d2131bf7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.530114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dcf4d-37af-4daa-8451-f6cabb2cc9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.531375 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dcf4d-37af-4daa-8451-f6cabb2cc9c0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.531818 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dcf4d-37af-4daa-8451-f6cabb2cc9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.532227 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dcf4d-37af-4daa-8451-f6cabb2cc9c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.533452 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7dcf4d-37af-4daa-8451-f6cabb2cc9c0 None admin] RBAC: Adding query filter params (role.id=44e187abae1d4b248fd145ad9cb97ed2, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=46cce59255af494eaa238ab8c8cddc51) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:49.533933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7dcf4d-37af-4daa-8451-f6cabb2cc9c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.536384 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7dcf4d-37af-4daa-8451-f6cabb2cc9c0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.551528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 438/864] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/role_assignments?user.id=46cce59255af494eaa238ab8c8cddc51&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=44e187abae1d4b248fd145ad9cb97ed2 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:49.611332 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb231de4-a008-4c40-a082-416616b9d03d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.651308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a00a3-e1db-40fb-b100-3acb57dfce48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.651952 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a00a3-e1db-40fb-b100-3acb57dfce48 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.652281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a00a3-e1db-40fb-b100-3acb57dfce48 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.652608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a00a3-e1db-40fb-b100-3acb57dfce48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.661341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a00a3-e1db-40fb-b100-3acb57dfce48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.662606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dcf4d-37af-4daa-8451-f6cabb2cc9c0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.663063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a00a3-e1db-40fb-b100-3acb57dfce48 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.681908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-379a00a3-e1db-40fb-b100-3acb57dfce48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:49.681908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:49.681908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:49.681908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:49.681908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:49.681908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:49.681908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:49.681908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:49.681908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:49.681908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:49.681908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:49.681908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:49.681908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:49.681908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:49.681908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:49.682965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:49.683842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:49.683842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:49.683842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:49.683842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:49.683842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:49.683842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:49.683842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:49.683842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:49.684270 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 427/865] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:49.690209 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a00a3-e1db-40fb-b100-3acb57dfce48 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.725726 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.746009 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.802873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dcf4d-37af-4daa-8451-f6cabb2cc9c0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.836676 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a00a3-e1db-40fb-b100-3acb57dfce48 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.857122 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75aa14-7a9d-4375-9b9a-4efff15552bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.858617 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75aa14-7a9d-4375-9b9a-4efff15552bc None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.859630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75aa14-7a9d-4375-9b9a-4efff15552bc None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.860178 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75aa14-7a9d-4375-9b9a-4efff15552bc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.860965 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75aa14-7a9d-4375-9b9a-4efff15552bc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.863182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75aa14-7a9d-4375-9b9a-4efff15552bc None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.879574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-aa75aa14-7a9d-4375-9b9a-4efff15552bc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:38:49.879574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:49.879574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:38:49.879574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:38:49.879574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:38:49.879574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:38:49.879574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:38:49.879574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:49.879574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:38:49.879574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:49.879574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:49.879574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:49.879574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:49.879574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:49.879574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:49.879574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:49.880609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:49.881515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:49.881515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:49.881515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:49.881515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:49.881515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:49.881515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:49.881515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:49.881515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:49.881515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:49.881515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:49.881515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:38:49.881515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:49.881515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:38:49.881515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:49.883598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 439/866] 10.222.0.91 () {56 vars in 1132 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:49.893839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75aa14-7a9d-4375-9b9a-4efff15552bc None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.895844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c35f6-c2f3-4b50-bc21-82bd21308d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.896529 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c35f6-c2f3-4b50-bc21-82bd21308d4e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.896799 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c35f6-c2f3-4b50-bc21-82bd21308d4e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.897072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c35f6-c2f3-4b50-bc21-82bd21308d4e None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.897821 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c35f6-c2f3-4b50-bc21-82bd21308d4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:49.898129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c35f6-c2f3-4b50-bc21-82bd21308d4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.899908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c35f6-c2f3-4b50-bc21-82bd21308d4e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.921966 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 428/867] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:49.928231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c35f6-c2f3-4b50-bc21-82bd21308d4e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:50.017215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75aa14-7a9d-4375-9b9a-4efff15552bc None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.025496 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:50.055780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249ad57-f70b-4cb7-98b4-1d4861897f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.056677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249ad57-f70b-4cb7-98b4-1d4861897f1c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.057310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249ad57-f70b-4cb7-98b4-1d4861897f1c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.057928 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249ad57-f70b-4cb7-98b4-1d4861897f1c None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.058832 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249ad57-f70b-4cb7-98b4-1d4861897f1c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:50.059338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249ad57-f70b-4cb7-98b4-1d4861897f1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:50.059516 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:50.061623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249ad57-f70b-4cb7-98b4-1d4861897f1c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.063227 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c35f6-c2f3-4b50-bc21-82bd21308d4e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.068015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 440/868] 10.222.0.91 () {56 vars in 1126 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:50.076938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249ad57-f70b-4cb7-98b4-1d4861897f1c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:50.109202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117fa52-72f2-424a-82fd-944d135526ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.110318 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117fa52-72f2-424a-82fd-944d135526ce None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.110946 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117fa52-72f2-424a-82fd-944d135526ce None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.111322 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117fa52-72f2-424a-82fd-944d135526ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.112844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117fa52-72f2-424a-82fd-944d135526ce None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=b3b5ae758fe847adaae0edaa0be095c9) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:50.113555 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117fa52-72f2-424a-82fd-944d135526ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:50.115392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117fa52-72f2-424a-82fd-944d135526ce None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.123363 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 429/869] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/role_assignments?user.id=b3b5ae758fe847adaae0edaa0be095c9&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:50.198113 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249ad57-f70b-4cb7-98b4-1d4861897f1c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.239960 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00913014-3a65-4886-aadf-3072677f083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.240647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00913014-3a65-4886-aadf-3072677f083d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.240923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00913014-3a65-4886-aadf-3072677f083d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.241193 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00913014-3a65-4886-aadf-3072677f083d None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.241823 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00913014-3a65-4886-aadf-3072677f083d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:50.242121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00913014-3a65-4886-aadf-3072677f083d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:50.243544 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00913014-3a65-4886-aadf-3072677f083d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.249189 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 441/870] 10.222.0.91 () {56 vars in 1126 bytes} [Thu Sep 8 13:38:50 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:51.227981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e4d4c-f6f4-48d0-b32a-f65f2d02d899 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:51.228537 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e4d4c-f6f4-48d0-b32a-f65f2d02d899 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:51.228807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e4d4c-f6f4-48d0-b32a-f65f2d02d899 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:51.230469 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 430/871] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:51.235709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5484f4-75df-4181-85e2-2fc90fc97e7b None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:51.236125 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5484f4-75df-4181-85e2-2fc90fc97e7b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:51.236301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5484f4-75df-4181-85e2-2fc90fc97e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:51.255183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-7b5484f4-75df-4181-85e2-2fc90fc97e7b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:51.256355 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7b5484f4-75df-4181-85e2-2fc90fc97e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:51.282930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:51.325707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 442/872] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:38:51.594839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e4d4c-f6f4-48d0-b32a-f65f2d02d899 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:51.711470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e4d4c-f6f4-48d0-b32a-f65f2d02d899 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:51.762085 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26f996-b7c2-4cae-94bb-dc0e7822526b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:51.763680 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26f996-b7c2-4cae-94bb-dc0e7822526b None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:51.764407 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26f996-b7c2-4cae-94bb-dc0e7822526b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:51.765119 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26f996-b7c2-4cae-94bb-dc0e7822526b None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:51.766535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26f996-b7c2-4cae-94bb-dc0e7822526b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:51.769161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26f996-b7c2-4cae-94bb-dc0e7822526b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:51.926401 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 431/873] 10.222.0.91 () {60 vars in 1141 bytes} [Thu Sep 8 13:38:51 2022] POST /identity/v3/services => generated 251 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:52.002661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70f877-65a9-44ad-b51f-0a970ce627a6 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.003880 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70f877-65a9-44ad-b51f-0a970ce627a6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.004370 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70f877-65a9-44ad-b51f-0a970ce627a6 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.006339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 443/874] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:52.013784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91efbea-1b59-429b-a889-0c31311c3fc0 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.014699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91efbea-1b59-429b-a889-0c31311c3fc0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.015600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91efbea-1b59-429b-a889-0c31311c3fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.044659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c91efbea-1b59-429b-a889-0c31311c3fc0 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:52.046331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c91efbea-1b59-429b-a889-0c31311c3fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:52.068735 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:52.133211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 432/875] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:52 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:38:52.408232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70f877-65a9-44ad-b51f-0a970ce627a6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:52.475717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b7f95-a522-4c05-836d-f4194958b655 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.476716 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b7f95-a522-4c05-836d-f4194958b655 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.477197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b7f95-a522-4c05-836d-f4194958b655 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.479624 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 433/876] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:52.485895 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b29af-8dee-4eda-b086-0037a1ff367b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.486454 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b29af-8dee-4eda-b086-0037a1ff367b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.486797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b29af-8dee-4eda-b086-0037a1ff367b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.507908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ca3b29af-8dee-4eda-b086-0037a1ff367b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:52.509447 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ca3b29af-8dee-4eda-b086-0037a1ff367b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:52.536081 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70f877-65a9-44ad-b51f-0a970ce627a6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:52.574397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d0953-f484-4d84-bb64-79d247427cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:52.575073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d0953-f484-4d84-bb64-79d247427cf7 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.575352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d0953-f484-4d84-bb64-79d247427cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.575622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d0953-f484-4d84-bb64-79d247427cf7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.587288 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d0953-f484-4d84-bb64-79d247427cf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:52.592167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-bd5d0953-f484-4d84-bb64-79d247427cf7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:38:52.592167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:52.592167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:52.592167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:52.592167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:52.592167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:52.592167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:52.592167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:52.592167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:52.592167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:52.592167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:52.592167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:52.592167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:52.592167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:52.592167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:52.592167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:52.593121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:52.593121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:52.593121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:52.593121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:52.593121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:52.593121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:52.593121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:52.593121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:52.593121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:52.593121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:52.593121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:52.593121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:52.593121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:52.593121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:52.593121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:52.593121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:52.594553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:52.594553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:52.594553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:52.594553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:52.594553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:52.594553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:52.594553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:52.594553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:52.594553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:52.594553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:52.594553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:38:52.594553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:52.595272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 434/877] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:52 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:38:52.595272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259d694-98af-4357-9012-dcc02267ce91 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.595272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259d694-98af-4357-9012-dcc02267ce91 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.595272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259d694-98af-4357-9012-dcc02267ce91 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.597059 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 435/878] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:52.599576 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 444/879] 10.222.0.91 () {56 vars in 1120 bytes} [Thu Sep 8 13:38:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:52.604932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259d694-98af-4357-9012-dcc02267ce91 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:52.606616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15549466-22fd-4331-acf1-a47718dd653f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.606959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15549466-22fd-4331-acf1-a47718dd653f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.607414 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15549466-22fd-4331-acf1-a47718dd653f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.627100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-15549466-22fd-4331-acf1-a47718dd653f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:52.628208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-15549466-22fd-4331-acf1-a47718dd653f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:52.648446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:52.710547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 445/880] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:52 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:38:52.723751 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:52.746364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259d694-98af-4357-9012-dcc02267ce91 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:52.785087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b0b34-b4cb-4da6-99ca-3e57be94fa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:52.785686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b0b34-b4cb-4da6-99ca-3e57be94fa20 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.786009 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b0b34-b4cb-4da6-99ca-3e57be94fa20 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.786278 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b0b34-b4cb-4da6-99ca-3e57be94fa20 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.786950 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b0b34-b4cb-4da6-99ca-3e57be94fa20 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:52.787250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b0b34-b4cb-4da6-99ca-3e57be94fa20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:52.788771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b0b34-b4cb-4da6-99ca-3e57be94fa20 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:52.797782 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 436/881] 10.222.0.91 () {56 vars in 1115 bytes} [Thu Sep 8 13:38:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:52.804006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15549466-22fd-4331-acf1-a47718dd653f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:52.864728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b0b34-b4cb-4da6-99ca-3e57be94fa20 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:52.925591 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15549466-22fd-4331-acf1-a47718dd653f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:52.964011 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92314c-025f-4c8b-8d7f-d976dfd3626f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:52.964557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92314c-025f-4c8b-8d7f-d976dfd3626f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.964833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92314c-025f-4c8b-8d7f-d976dfd3626f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.965104 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92314c-025f-4c8b-8d7f-d976dfd3626f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.977698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92314c-025f-4c8b-8d7f-d976dfd3626f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:52.979492 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd92314c-025f-4c8b-8d7f-d976dfd3626f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:52.995314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-bd92314c-025f-4c8b-8d7f-d976dfd3626f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:52.995314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:52.995314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:52.995314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:52.995314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:52.995314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:52.995314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:52.995314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:52.995314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:52.995314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:52.995314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:52.995314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:52.995314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:52.995314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:52.995314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:52.995314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:52.996296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:52.997174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:52.997174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:52.997174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:52.997174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 446/882] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:52.999713 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b0b34-b4cb-4da6-99ca-3e57be94fa20 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.003533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92314c-025f-4c8b-8d7f-d976dfd3626f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.013737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.039121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56386d1-ce33-4193-9e2f-9fcd2aae8c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.039751 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56386d1-ce33-4193-9e2f-9fcd2aae8c30 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.040030 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56386d1-ce33-4193-9e2f-9fcd2aae8c30 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.040324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56386d1-ce33-4193-9e2f-9fcd2aae8c30 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.040906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56386d1-ce33-4193-9e2f-9fcd2aae8c30 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.042317 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56386d1-ce33-4193-9e2f-9fcd2aae8c30 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-c56386d1-ce33-4193-9e2f-9fcd2aae8c30 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:53.052380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:53.053387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:53.053387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:53.053387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:53.053387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:53.053387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:53.053387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:53.053387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:38:53.053387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:38:53.053387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:38:53.053387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:38:53.053387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:53.053387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:53.053387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:53.053387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:53.053387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:53.053387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:53.054196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:53.054196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:53.054196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:53.054196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:53.054196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:53.054196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:53.054196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:53.054196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:53.054196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:53.054196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:53.054196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:38:53.054196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:53.054196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:38:53.054196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:53.054196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 437/883] 10.222.0.91 () {56 vars in 1126 bytes} [Thu Sep 8 13:38:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:53.055017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56386d1-ce33-4193-9e2f-9fcd2aae8c30 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.067330 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.123474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd92314c-025f-4c8b-8d7f-d976dfd3626f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.160343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd2c20-189f-4184-8991-c534688eb131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.161327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd2c20-189f-4184-8991-c534688eb131 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.161798 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd2c20-189f-4184-8991-c534688eb131 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.162210 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd2c20-189f-4184-8991-c534688eb131 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.163018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd2c20-189f-4184-8991-c534688eb131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:53.163460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd2c20-189f-4184-8991-c534688eb131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.165143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd2c20-189f-4184-8991-c534688eb131 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.167537 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56386d1-ce33-4193-9e2f-9fcd2aae8c30 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.179343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 447/884] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:53.180445 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd2c20-189f-4184-8991-c534688eb131 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.208223 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9ae3a-7a94-4c15-bf5b-5a9b8df7b043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.209285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9ae3a-7a94-4c15-bf5b-5a9b8df7b043 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.209559 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9ae3a-7a94-4c15-bf5b-5a9b8df7b043 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.209834 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9ae3a-7a94-4c15-bf5b-5a9b8df7b043 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.210757 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9ae3a-7a94-4c15-bf5b-5a9b8df7b043 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.213151 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9ae3a-7a94-4c15-bf5b-5a9b8df7b043 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.247186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.281803 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 438/885] 10.222.0.91 () {60 vars in 1141 bytes} [Thu Sep 8 13:38:53 2022] POST /identity/v3/services => generated 269 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:53.283498 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9ae3a-7a94-4c15-bf5b-5a9b8df7b043 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.285120 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd2c20-189f-4184-8991-c534688eb131 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.347372 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c0f48-fde9-43e3-9ae1-7de2aaaf4def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.348001 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c0f48-fde9-43e3-9ae1-7de2aaaf4def None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.348273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c0f48-fde9-43e3-9ae1-7de2aaaf4def None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.348580 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c0f48-fde9-43e3-9ae1-7de2aaaf4def None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.349223 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c0f48-fde9-43e3-9ae1-7de2aaaf4def None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:53.349518 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c0f48-fde9-43e3-9ae1-7de2aaaf4def None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.350965 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c0f48-fde9-43e3-9ae1-7de2aaaf4def None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.362154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 448/886] 10.222.0.91 () {56 vars in 1122 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:53.363295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c0f48-fde9-43e3-9ae1-7de2aaaf4def None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.397785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.449230 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9ae3a-7a94-4c15-bf5b-5a9b8df7b043 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.495200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab60b8-49a3-489f-aa6d-d69a75dc13f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.495712 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c0f48-fde9-43e3-9ae1-7de2aaaf4def None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.496058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab60b8-49a3-489f-aa6d-d69a75dc13f3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.496058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab60b8-49a3-489f-aa6d-d69a75dc13f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.496058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab60b8-49a3-489f-aa6d-d69a75dc13f3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.507671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab60b8-49a3-489f-aa6d-d69a75dc13f3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.509074 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab60b8-49a3-489f-aa6d-d69a75dc13f3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.525232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-deab60b8-49a3-489f-aa6d-d69a75dc13f3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:38:53.525232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:53.525232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:53.525232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:53.525232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:53.525232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:53.525232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:53.525232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:53.525232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:53.525232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:53.525232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:53.525232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:53.525232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:53.525232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:53.525232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:53.525232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:53.527505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:53.528429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:53.528429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:53.528429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:53.528429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:53.528429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:53.528429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:53.528429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:53.528429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:53.528429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:38:53.528429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:53.528429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 439/887] 10.222.0.91 () {56 vars in 1117 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:53.529622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.532542 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab60b8-49a3-489f-aa6d-d69a75dc13f3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.543618 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59592de-0984-4cb1-8db5-4ae07900f8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.544710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59592de-0984-4cb1-8db5-4ae07900f8d2 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.544817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59592de-0984-4cb1-8db5-4ae07900f8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.545571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59592de-0984-4cb1-8db5-4ae07900f8d2 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.546222 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59592de-0984-4cb1-8db5-4ae07900f8d2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:53.546529 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59592de-0984-4cb1-8db5-4ae07900f8d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.548359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59592de-0984-4cb1-8db5-4ae07900f8d2 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.554386 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 449/888] 10.222.0.91 () {56 vars in 1122 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:53.660830 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab60b8-49a3-489f-aa6d-d69a75dc13f3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.690456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.701154 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceab94f-5fda-46f1-947e-3cf448794764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.701819 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceab94f-5fda-46f1-947e-3cf448794764 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.702099 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceab94f-5fda-46f1-947e-3cf448794764 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.702376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceab94f-5fda-46f1-947e-3cf448794764 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.707228 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceab94f-5fda-46f1-947e-3cf448794764 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:53.707228 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceab94f-5fda-46f1-947e-3cf448794764 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.707228 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceab94f-5fda-46f1-947e-3cf448794764 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.727524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 440/889] 10.222.0.91 () {56 vars in 1149 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:53.733346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59592de-0984-4cb1-8db5-4ae07900f8d2 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.847457 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59592de-0984-4cb1-8db5-4ae07900f8d2 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.885408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea933a-bee0-410c-abad-19944f404df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.886086 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea933a-bee0-410c-abad-19944f404df4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.886369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea933a-bee0-410c-abad-19944f404df4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.886639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea933a-bee0-410c-abad-19944f404df4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.954027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.965729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea933a-bee0-410c-abad-19944f404df4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.967556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea933a-bee0-410c-abad-19944f404df4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:54.107312 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 450/890] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:38:54.115681 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceab94f-5fda-46f1-947e-3cf448794764 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.275869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d5acb-a28b-4d7c-893e-f7c6d8ce818a None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.276156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d5acb-a28b-4d7c-893e-f7c6d8ce818a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.276483 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d5acb-a28b-4d7c-893e-f7c6d8ce818a None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.277785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 451/891] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:54.283355 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60509b7-fc23-4e9f-8a10-1fd119ea98b0 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.283905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60509b7-fc23-4e9f-8a10-1fd119ea98b0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.284949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60509b7-fc23-4e9f-8a10-1fd119ea98b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.298653 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceab94f-5fda-46f1-947e-3cf448794764 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:54.305002 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d60509b7-fc23-4e9f-8a10-1fd119ea98b0 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:54.306496 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d60509b7-fc23-4e9f-8a10-1fd119ea98b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:54.377270 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c381b6-beec-44d4-a7da-ba4a6d796e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:54.379201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c381b6-beec-44d4-a7da-ba4a6d796e06 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.380180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c381b6-beec-44d4-a7da-ba4a6d796e06 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.380718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c381b6-beec-44d4-a7da-ba4a6d796e06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.392880 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 452/892] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:38:54.403315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c381b6-beec-44d4-a7da-ba4a6d796e06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:54.404880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c381b6-beec-44d4-a7da-ba4a6d796e06 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:54.425413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-67c381b6-beec-44d4-a7da-ba4a6d796e06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:54.425413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:54.425413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:54.425413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:54.425413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:54.425413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:54.425413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:54.425413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:54.425413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:54.425413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:54.425413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:54.425413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:54.425413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:54.425413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:54.425413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:54.425413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:54.426406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:54.427316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:54.427316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:54.427316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:54.427316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 441/893] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:54.432591 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60509b7-fc23-4e9f-8a10-1fd119ea98b0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.458281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:54.545281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60509b7-fc23-4e9f-8a10-1fd119ea98b0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:54.589259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc5d60-1f12-43e3-8a74-42e97fcf9a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:54.590328 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc5d60-1f12-43e3-8a74-42e97fcf9a04 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.590807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc5d60-1f12-43e3-8a74-42e97fcf9a04 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.591312 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc5d60-1f12-43e3-8a74-42e97fcf9a04 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.592467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc5d60-1f12-43e3-8a74-42e97fcf9a04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:54.592981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc5d60-1f12-43e3-8a74-42e97fcf9a04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:54.595394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc5d60-1f12-43e3-8a74-42e97fcf9a04 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:54.616748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 453/894] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:54.622451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c381b6-beec-44d4-a7da-ba4a6d796e06 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.661423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:54.683060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc5d60-1f12-43e3-8a74-42e97fcf9a04 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.752311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c381b6-beec-44d4-a7da-ba4a6d796e06 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:54.787398 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:54.794919 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdd234-00fb-4fe6-81e9-0a09f159845c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:54.796276 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdd234-00fb-4fe6-81e9-0a09f159845c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.797127 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdd234-00fb-4fe6-81e9-0a09f159845c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.797612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdd234-00fb-4fe6-81e9-0a09f159845c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.806503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bdd234-00fb-4fe6-81e9-0a09f159845c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:54.808260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bdd234-00fb-4fe6-81e9-0a09f159845c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:54.815011 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc5d60-1f12-43e3-8a74-42e97fcf9a04 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:54.822260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-91bdd234-00fb-4fe6-81e9-0a09f159845c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:54.822260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:54.822260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:54.822260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:54.822260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:54.822260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:54.822260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:54.822260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:54.822260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:54.822260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:54.822260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:54.822260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:54.822260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:54.822260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:54.822260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:54.823315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:54.824293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:54.824293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:54.824293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:54.824293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:54.824293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:54.824293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:54.824293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:54.824293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:54.827018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 442/895] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:54.832511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdd234-00fb-4fe6-81e9-0a09f159845c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.863462 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc03c36-ceeb-4d28-bfae-b71260a47b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:54.864100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc03c36-ceeb-4d28-bfae-b71260a47b2b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.864598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc03c36-ceeb-4d28-bfae-b71260a47b2b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.864883 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc03c36-ceeb-4d28-bfae-b71260a47b2b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.865517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc03c36-ceeb-4d28-bfae-b71260a47b2b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:54.867612 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc03c36-ceeb-4d28-bfae-b71260a47b2b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-bcc03c36-ceeb-4d28-bfae-b71260a47b2b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:54.879653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:54.880709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:54.880709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:54.880709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:54.880709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:54.880709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:54.880709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:54.880709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:38:54.880709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:38:54.880709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:38:54.880709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:38:54.880709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:54.880709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:54.880709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:54.880709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:54.880709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:54.880709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:54.881567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:54.881567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:54.881567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:54.881567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:54.881567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:54.881567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:54.881567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:54.881567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:54.881567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:54.881567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:54.881567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:38:54.881567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:54.881567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:38:54.881567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:54.881567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 454/896] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:54.887131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc03c36-ceeb-4d28-bfae-b71260a47b2b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.974230 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:54.980816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdd234-00fb-4fe6-81e9-0a09f159845c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.022101 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b0ae3-796c-4352-961a-fae8856fdd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.022827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b0ae3-796c-4352-961a-fae8856fdd47 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.023211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b0ae3-796c-4352-961a-fae8856fdd47 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.023596 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b0ae3-796c-4352-961a-fae8856fdd47 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.024348 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b0ae3-796c-4352-961a-fae8856fdd47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:55.024719 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b0ae3-796c-4352-961a-fae8856fdd47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.026319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b0ae3-796c-4352-961a-fae8856fdd47 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.044218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 443/897] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:55.046355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b0ae3-796c-4352-961a-fae8856fdd47 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:55.064088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc03c36-ceeb-4d28-bfae-b71260a47b2b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.117231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945fd10-8406-4be6-91bd-ffde4c29a532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.118411 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945fd10-8406-4be6-91bd-ffde4c29a532 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.119339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945fd10-8406-4be6-91bd-ffde4c29a532 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.119936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6945fd10-8406-4be6-91bd-ffde4c29a532 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.120864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945fd10-8406-4be6-91bd-ffde4c29a532 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:55.121360 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945fd10-8406-4be6-91bd-ffde4c29a532 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.123208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6945fd10-8406-4be6-91bd-ffde4c29a532 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.130783 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 455/898] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:55.135995 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945fd10-8406-4be6-91bd-ffde4c29a532 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:55.153430 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:55.163014 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b0ae3-796c-4352-961a-fae8856fdd47 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.202848 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27aac6b-f381-4934-8247-bb044a1cc112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.203577 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27aac6b-f381-4934-8247-bb044a1cc112 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.203863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27aac6b-f381-4934-8247-bb044a1cc112 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.204406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27aac6b-f381-4934-8247-bb044a1cc112 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.204846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27aac6b-f381-4934-8247-bb044a1cc112 None admin] RBAC: Adding query filter params (role.id=cf5de1e5771c4b60911d58b68de13600, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=46cce59255af494eaa238ab8c8cddc51) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:55.205208 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27aac6b-f381-4934-8247-bb044a1cc112 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.207204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27aac6b-f381-4934-8247-bb044a1cc112 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.222928 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 444/899] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:55 2022] GET /identity/v3/role_assignments?user.id=46cce59255af494eaa238ab8c8cddc51&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=cf5de1e5771c4b60911d58b68de13600 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:55.282082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6945fd10-8406-4be6-91bd-ffde4c29a532 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.334318 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f9c04-c40b-4f6c-a77f-cfca1d434882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.335427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f9c04-c40b-4f6c-a77f-cfca1d434882 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.335950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f9c04-c40b-4f6c-a77f-cfca1d434882 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.336490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f9c04-c40b-4f6c-a77f-cfca1d434882 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.337388 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f9c04-c40b-4f6c-a77f-cfca1d434882 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:55.337935 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f9c04-c40b-4f6c-a77f-cfca1d434882 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.339602 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f9c04-c40b-4f6c-a77f-cfca1d434882 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.346560 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 456/900] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:55.353239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27aac6b-f381-4934-8247-bb044a1cc112 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:55.381074 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:55.473496 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27aac6b-f381-4934-8247-bb044a1cc112 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.515689 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8815a-17a6-4035-b52b-b0c78e087cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.516695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8815a-17a6-4035-b52b-b0c78e087cf0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.517179 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8815a-17a6-4035-b52b-b0c78e087cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.517675 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8815a-17a6-4035-b52b-b0c78e087cf0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.518628 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8815a-17a6-4035-b52b-b0c78e087cf0 None admin] RBAC: Adding query filter params (interface=public, service_id=2f516fb0e9ac43ec83f0ce060878eada, region_id=RegionOne) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:55.519645 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8815a-17a6-4035-b52b-b0c78e087cf0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.521235 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8815a-17a6-4035-b52b-b0c78e087cf0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.529631 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 445/901] 10.222.0.91 () {56 vars in 1251 bytes} [Thu Sep 8 13:38:55 2022] GET /identity/v3/endpoints?service_id=2f516fb0e9ac43ec83f0ce060878eada&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:55.535652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f9c04-c40b-4f6c-a77f-cfca1d434882 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:55.614378 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:55.650632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f9c04-c40b-4f6c-a77f-cfca1d434882 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.692319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ae6b0-596e-48ee-aa6a-fed1bb16ed2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.693311 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ae6b0-596e-48ee-aa6a-fed1bb16ed2c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.693796 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ae6b0-596e-48ee-aa6a-fed1bb16ed2c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.694276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ae6b0-596e-48ee-aa6a-fed1bb16ed2c None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.695116 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ae6b0-596e-48ee-aa6a-fed1bb16ed2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.696792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ae6b0-596e-48ee-aa6a-fed1bb16ed2c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.703767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 457/902] 10.222.0.91 () {56 vars in 1087 bytes} [Thu Sep 8 13:38:55 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:38:55.776903 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448099a0-02ac-41d2-b0f9-95a62c2de0bc None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.777902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448099a0-02ac-41d2-b0f9-95a62c2de0bc None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.778394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448099a0-02ac-41d2-b0f9-95a62c2de0bc None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.780139 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 446/903] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:55.786415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199a34b-95fe-4e92-acaf-022963442bf8 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.787058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199a34b-95fe-4e92-acaf-022963442bf8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.787546 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199a34b-95fe-4e92-acaf-022963442bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.807765 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-0199a34b-95fe-4e92-acaf-022963442bf8 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:55.809365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0199a34b-95fe-4e92-acaf-022963442bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:55.892541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 458/904] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:38:55.951697 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2102d-322c-4399-8891-f1c471291656 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.952331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2102d-322c-4399-8891-f1c471291656 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.952813 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2102d-322c-4399-8891-f1c471291656 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.954630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 447/905] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:55.960035 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1d96a-597d-4ec5-91b3-bbbb97e55dd1 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.960321 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1d96a-597d-4ec5-91b3-bbbb97e55dd1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.960598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1d96a-597d-4ec5-91b3-bbbb97e55dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.980170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d5c1d96a-597d-4ec5-91b3-bbbb97e55dd1 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:55.981521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d5c1d96a-597d-4ec5-91b3-bbbb97e55dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:56.045037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:56.053351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 459/906] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:38:56.175511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2102d-322c-4399-8891-f1c471291656 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:56.267537 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:56.285080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2102d-322c-4399-8891-f1c471291656 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:56.320747 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434f293-83bc-4170-9533-884209c0c3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:56.321390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434f293-83bc-4170-9533-884209c0c3c7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.321632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434f293-83bc-4170-9533-884209c0c3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.321913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434f293-83bc-4170-9533-884209c0c3c7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.322499 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434f293-83bc-4170-9533-884209c0c3c7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:56.324294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434f293-83bc-4170-9533-884209c0c3c7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:56.334187 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1d96a-597d-4ec5-91b3-bbbb97e55dd1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:56.334948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-b434f293-83bc-4170-9533-884209c0c3c7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:38:56.334948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:56.334948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:38:56.334948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:38:56.334948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:38:56.334948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:38:56.334948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:38:56.334948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:56.334948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:38:56.334948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:56.334948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:56.334948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:56.334948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:56.334948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:56.334948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:56.334948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:56.335845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:56.336711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:56.336711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:56.336711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:56.336711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:56.336711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:56.336711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:56.336711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:56.336711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:56.336711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:56.336711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:56.336711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:38:56.336711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:56.336711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:38:56.336711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:56.336711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 448/907] 10.222.0.91 () {56 vars in 1132 bytes} [Thu Sep 8 13:38:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:56.343194 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434f293-83bc-4170-9533-884209c0c3c7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:56.447959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1d96a-597d-4ec5-91b3-bbbb97e55dd1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:56.477767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434f293-83bc-4170-9533-884209c0c3c7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:56.493885 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4371b-8c06-40e6-a8db-ea3896f8acad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:56.494772 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4371b-8c06-40e6-a8db-ea3896f8acad None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.495220 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4371b-8c06-40e6-a8db-ea3896f8acad None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.495643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4371b-8c06-40e6-a8db-ea3896f8acad None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.496466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4371b-8c06-40e6-a8db-ea3896f8acad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:56.498033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4371b-8c06-40e6-a8db-ea3896f8acad None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:56.514120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756ff4f-b403-435a-a5d4-e21e4b5b3b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:56.514722 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756ff4f-b403-435a-a5d4-e21e4b5b3b94 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.515020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756ff4f-b403-435a-a5d4-e21e4b5b3b94 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.515285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756ff4f-b403-435a-a5d4-e21e4b5b3b94 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.515904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756ff4f-b403-435a-a5d4-e21e4b5b3b94 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:56.516238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756ff4f-b403-435a-a5d4-e21e4b5b3b94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:56.517586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756ff4f-b403-435a-a5d4-e21e4b5b3b94 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:56.522586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 449/908] 10.222.0.91 () {56 vars in 1126 bytes} [Thu Sep 8 13:38:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:56.528172 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756ff4f-b403-435a-a5d4-e21e4b5b3b94 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:56.544309 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:56.580197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 460/909] 10.222.0.91 () {60 vars in 1141 bytes} [Thu Sep 8 13:38:56 2022] POST /identity/v3/services => generated 241 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:56.640053 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756ff4f-b403-435a-a5d4-e21e4b5b3b94 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:56.686169 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1a735-254e-4b06-b94d-734e02289a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:56.686774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1a735-254e-4b06-b94d-734e02289a21 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.687096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1a735-254e-4b06-b94d-734e02289a21 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.687360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1a735-254e-4b06-b94d-734e02289a21 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.687988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a1a735-254e-4b06-b94d-734e02289a21 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:56.688287 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a1a735-254e-4b06-b94d-734e02289a21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:56.689653 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a1a735-254e-4b06-b94d-734e02289a21 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:56.699080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 450/910] 10.222.0.91 () {56 vars in 1126 bytes} [Thu Sep 8 13:38:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:56.701947 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4371b-8c06-40e6-a8db-ea3896f8acad None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:56.768174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:56.814490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4371b-8c06-40e6-a8db-ea3896f8acad None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:56.849636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722fd097-fadc-41a1-b1e3-1108b6fced54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:56.850155 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722fd097-fadc-41a1-b1e3-1108b6fced54 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.850429 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722fd097-fadc-41a1-b1e3-1108b6fced54 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.850695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722fd097-fadc-41a1-b1e3-1108b6fced54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.851410 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722fd097-fadc-41a1-b1e3-1108b6fced54 None admin] RBAC: Adding query filter params (interface=public, service_id=5854933404444aebbf9c72dcc3801750, region_id=RegionOne) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:56.851712 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722fd097-fadc-41a1-b1e3-1108b6fced54 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:56.853070 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722fd097-fadc-41a1-b1e3-1108b6fced54 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:56.858961 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 461/911] 10.222.0.91 () {56 vars in 1251 bytes} [Thu Sep 8 13:38:56 2022] GET /identity/v3/endpoints?service_id=5854933404444aebbf9c72dcc3801750&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:56.864904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1a735-254e-4b06-b94d-734e02289a21 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:56.944459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:56.967531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1a735-254e-4b06-b94d-734e02289a21 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:57.002959 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f63254-b2e1-4650-899c-f57f8dccb3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:57.004226 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f63254-b2e1-4650-899c-f57f8dccb3cf None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:57.004498 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f63254-b2e1-4650-899c-f57f8dccb3cf None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:57.004763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f63254-b2e1-4650-899c-f57f8dccb3cf None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:57.005361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f63254-b2e1-4650-899c-f57f8dccb3cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:57.006734 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f63254-b2e1-4650-899c-f57f8dccb3cf None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:57.019448 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 451/912] 10.222.0.91 () {56 vars in 1087 bytes} [Thu Sep 8 13:38:56 2022] GET /identity/v3/services => generated 1497 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:38:57.516785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63263fc-23d4-4687-991c-46811d2aaa5c None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:57.517424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63263fc-23d4-4687-991c-46811d2aaa5c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:57.517924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63263fc-23d4-4687-991c-46811d2aaa5c None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:57.520300 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 462/913] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:57.526255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0996f4-6a60-4ca7-824b-7fa8d1c31df0 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:57.527090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0996f4-6a60-4ca7-824b-7fa8d1c31df0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:57.527638 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0996f4-6a60-4ca7-824b-7fa8d1c31df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:57.548100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0f0996f4-6a60-4ca7-824b-7fa8d1c31df0 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:57.549286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0f0996f4-6a60-4ca7-824b-7fa8d1c31df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:57.617917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 452/914] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:38:57.894578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63263fc-23d4-4687-991c-46811d2aaa5c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:57.959312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a5816-ce31-49a1-886e-6228028758b6 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:57.960296 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a5816-ce31-49a1-886e-6228028758b6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:57.960847 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a5816-ce31-49a1-886e-6228028758b6 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:57.962785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 453/915] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:57.968049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd251d29-db73-4631-9664-168e65d8bc02 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:57.968402 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd251d29-db73-4631-9664-168e65d8bc02 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:57.968750 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd251d29-db73-4631-9664-168e65d8bc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:57.989920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-cd251d29-db73-4631-9664-168e65d8bc02 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:57.991507 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-cd251d29-db73-4631-9664-168e65d8bc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:58.011826 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63263fc-23d4-4687-991c-46811d2aaa5c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:58.033944 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:58.048097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477b222-81bd-4dd8-995a-68365039357f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:58.048976 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477b222-81bd-4dd8-995a-68365039357f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.054292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477b222-81bd-4dd8-995a-68365039357f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.054292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477b222-81bd-4dd8-995a-68365039357f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.060377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477b222-81bd-4dd8-995a-68365039357f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:58.062967 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:58.066937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-2477b222-81bd-4dd8-995a-68365039357f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:38:58.066937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:58.066937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:58.066937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:58.066937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:58.066937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:58.066937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:58.066937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:58.066937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:58.066937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:58.066937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:58.066937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:58.066937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:58.066937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:58.066937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:58.066937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:58.068082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:58.068082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:58.068082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:58.068082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:58.068082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:58.068082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:58.068082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:58.068082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:58.068082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:58.068082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:58.068082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:58.068082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:58.068082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:58.068082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:58.068082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:58.068082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:58.069027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:58.069027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:58.069027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:58.069027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:58.069027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:58.069027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:58.069027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:58.069027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:58.069027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:58.069027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:58.069027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:38:58.069027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:58.071524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 463/916] 10.222.0.91 () {56 vars in 1120 bytes} [Thu Sep 8 13:38:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:58.073295 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 454/917] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:38:58.079938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477b222-81bd-4dd8-995a-68365039357f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:58.186047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477b222-81bd-4dd8-995a-68365039357f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:58.222697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0394c-e211-42b0-8be9-cc8137f0f39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:58.223378 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0394c-e211-42b0-8be9-cc8137f0f39a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.223655 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0394c-e211-42b0-8be9-cc8137f0f39a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.223926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0394c-e211-42b0-8be9-cc8137f0f39a None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.224578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0394c-e211-42b0-8be9-cc8137f0f39a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:58.225050 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0394c-e211-42b0-8be9-cc8137f0f39a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:58.226436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0394c-e211-42b0-8be9-cc8137f0f39a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:58.236799 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 464/918] 10.222.0.91 () {56 vars in 1115 bytes} [Thu Sep 8 13:38:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:58.244205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd251d29-db73-4631-9664-168e65d8bc02 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:58.348061 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd251d29-db73-4631-9664-168e65d8bc02 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:58.371711 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0394c-e211-42b0-8be9-cc8137f0f39a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:58.385319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1213b-b2d4-488c-99ef-1401a08cec93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:58.386022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1213b-b2d4-488c-99ef-1401a08cec93 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.386301 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1213b-b2d4-488c-99ef-1401a08cec93 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.387008 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1213b-b2d4-488c-99ef-1401a08cec93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.399241 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1213b-b2d4-488c-99ef-1401a08cec93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:58.400697 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1213b-b2d4-488c-99ef-1401a08cec93 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:58.421891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-e1d1213b-b2d4-488c-99ef-1401a08cec93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:58.421891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:58.421891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:58.421891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:58.421891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:58.421891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:58.421891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:58.421891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:58.421891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:58.421891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:58.421891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:58.421891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:58.421891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:58.421891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:58.421891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:58.421891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:58.422874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:58.423810 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:58.423810 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:58.423810 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:58.424049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:58.426441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 455/919] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:58.435026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1213b-b2d4-488c-99ef-1401a08cec93 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:58.497810 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0394c-e211-42b0-8be9-cc8137f0f39a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:58.503757 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:58.541535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4cd46-1871-41be-af93-2a778b8bb7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:58.543157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4cd46-1871-41be-af93-2a778b8bb7e9 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.543624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4cd46-1871-41be-af93-2a778b8bb7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.544354 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4cd46-1871-41be-af93-2a778b8bb7e9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.545119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4cd46-1871-41be-af93-2a778b8bb7e9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:58.547169 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4cd46-1871-41be-af93-2a778b8bb7e9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-98e4cd46-1871-41be-af93-2a778b8bb7e9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:58.558984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:58.559969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:58.559969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:58.559969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:58.559969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:58.559969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:58.559969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:58.559969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:38:58.559969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:38:58.559969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:38:58.559969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:38:58.559969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:58.559969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:58.559969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:58.559969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:58.559969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:58.559969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:58.560806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:58.560806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:58.560806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:58.560806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:58.560806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:58.560806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:58.560806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:58.560806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:58.560806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:58.560806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:58.560806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:38:58.560806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:58.560806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:38:58.560806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:58.563620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 465/920] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:58.564933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1213b-b2d4-488c-99ef-1401a08cec93 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:58.574950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4cd46-1871-41be-af93-2a778b8bb7e9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:58.610801 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ebc46-e794-4dd4-ae64-19f19850818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:58.611548 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ebc46-e794-4dd4-ae64-19f19850818d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.611896 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ebc46-e794-4dd4-ae64-19f19850818d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.612248 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ebc46-e794-4dd4-ae64-19f19850818d None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.613638 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ebc46-e794-4dd4-ae64-19f19850818d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:58.614028 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ebc46-e794-4dd4-ae64-19f19850818d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:58.615521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ebc46-e794-4dd4-ae64-19f19850818d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:58.635040 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 456/921] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:58.637418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ebc46-e794-4dd4-ae64-19f19850818d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:58.703154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4cd46-1871-41be-af93-2a778b8bb7e9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:58.715942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:58.763080 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33a69e-4038-4677-a57d-ec891b2fb35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:58.763608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33a69e-4038-4677-a57d-ec891b2fb35d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.763881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33a69e-4038-4677-a57d-ec891b2fb35d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.764152 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33a69e-4038-4677-a57d-ec891b2fb35d None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.764386 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ebc46-e794-4dd4-ae64-19f19850818d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:58.764846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33a69e-4038-4677-a57d-ec891b2fb35d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:58.765152 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33a69e-4038-4677-a57d-ec891b2fb35d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:58.766824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a33a69e-4038-4677-a57d-ec891b2fb35d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:58.778880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:58.779081 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 466/922] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:58.780430 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33a69e-4038-4677-a57d-ec891b2fb35d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:58.807460 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38703b6e-92f7-422f-a076-68ee29495cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:58.808466 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38703b6e-92f7-422f-a076-68ee29495cfd None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.809040 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38703b6e-92f7-422f-a076-68ee29495cfd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.809698 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38703b6e-92f7-422f-a076-68ee29495cfd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.818148 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38703b6e-92f7-422f-a076-68ee29495cfd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:58.819953 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38703b6e-92f7-422f-a076-68ee29495cfd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:58.838032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-38703b6e-92f7-422f-a076-68ee29495cfd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:38:58.838032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:58.838032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:58.838032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:58.838032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:58.838032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:58.838032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:58.838032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:58.838032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:58.838032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:58.838032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:58.838032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:58.838032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:58.838032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:58.838032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:58.838032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:58.839445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:58.840405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:58.840405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:58.840405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:58.840405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:58.840405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:58.840405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:58.840405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:58.840405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:58.840405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:38:58.840405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:58.840405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 457/923] 10.222.0.91 () {56 vars in 1117 bytes} [Thu Sep 8 13:38:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:58.841028 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ce2f9-7486-4a76-9054-4c50b3aaa22c None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.841028 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ce2f9-7486-4a76-9054-4c50b3aaa22c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.841150 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ce2f9-7486-4a76-9054-4c50b3aaa22c None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.842469 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 458/924] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:58.848138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ce2f9-7486-4a76-9054-4c50b3aaa22c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:58.902351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a33a69e-4038-4677-a57d-ec891b2fb35d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:58.942653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901dc6b-ab49-4848-a9ae-eb24610e435d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:58.943301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901dc6b-ab49-4848-a9ae-eb24610e435d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.943604 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901dc6b-ab49-4848-a9ae-eb24610e435d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.943846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901dc6b-ab49-4848-a9ae-eb24610e435d None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.944474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901dc6b-ab49-4848-a9ae-eb24610e435d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:58.944771 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901dc6b-ab49-4848-a9ae-eb24610e435d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:58.946273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901dc6b-ab49-4848-a9ae-eb24610e435d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:58.956827 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 467/925] 10.222.0.91 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:58.956827 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abf811-c500-413e-a2c0-e804bf55893c None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.956827 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abf811-c500-413e-a2c0-e804bf55893c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.956827 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abf811-c500-413e-a2c0-e804bf55893c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.966209 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ce2f9-7486-4a76-9054-4c50b3aaa22c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:58.973452 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-59abf811-c500-413e-a2c0-e804bf55893c None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:58.974647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-59abf811-c500-413e-a2c0-e804bf55893c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:59.000960 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:59.003424 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be71b8-114c-4696-ae6d-dba7a3eb935e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:59.004245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be71b8-114c-4696-ae6d-dba7a3eb935e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.004518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be71b8-114c-4696-ae6d-dba7a3eb935e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.004784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be71b8-114c-4696-ae6d-dba7a3eb935e None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.005512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be71b8-114c-4696-ae6d-dba7a3eb935e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:59.005815 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be71b8-114c-4696-ae6d-dba7a3eb935e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:59.007285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be71b8-114c-4696-ae6d-dba7a3eb935e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:59.027812 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 459/926] 10.222.0.91 () {56 vars in 1149 bytes} [Thu Sep 8 13:38:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:59.033877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be71b8-114c-4696-ae6d-dba7a3eb935e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:59.049737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 468/927] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:38:59.055903 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59abf811-c500-413e-a2c0-e804bf55893c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:59.067143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:59.153845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be71b8-114c-4696-ae6d-dba7a3eb935e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:59.197309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59abf811-c500-413e-a2c0-e804bf55893c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:59.198943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497e1a8-d65a-4332-afdc-4313efe10c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:59.199682 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497e1a8-d65a-4332-afdc-4313efe10c08 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.199862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497e1a8-d65a-4332-afdc-4313efe10c08 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.200108 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497e1a8-d65a-4332-afdc-4313efe10c08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.200775 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497e1a8-d65a-4332-afdc-4313efe10c08 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:59.202374 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497e1a8-d65a-4332-afdc-4313efe10c08 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:59.238366 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e1ef6-7a7e-4f85-b68c-700e53d30e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:59.239416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e1ef6-7a7e-4f85-b68c-700e53d30e83 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.239826 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e1ef6-7a7e-4f85-b68c-700e53d30e83 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.240265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e1ef6-7a7e-4f85-b68c-700e53d30e83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.251571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e1ef6-7a7e-4f85-b68c-700e53d30e83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:59.253806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e1ef6-7a7e-4f85-b68c-700e53d30e83 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:59.271162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-8e6e1ef6-7a7e-4f85-b68c-700e53d30e83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:59.271162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:59.271162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:59.271162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:59.271162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:59.271162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:59.271162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:59.271162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:59.271162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:59.271162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:59.271162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:59.271162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:59.271162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:59.271162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:59.271162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:59.271162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:59.272240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:59.273096 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:59.273096 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:59.273096 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:59.275045 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 469/928] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:59.278728 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e1ef6-7a7e-4f85-b68c-700e53d30e83 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:59.290468 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 460/929] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:38:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:59.292312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e7146-252e-429c-b079-536b8826700b None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.294157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e7146-252e-429c-b079-536b8826700b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.294323 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e7146-252e-429c-b079-536b8826700b None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.296253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 461/930] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:59.304348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:59.308691 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367d899-117d-4a4b-9cdf-285641c986ab None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.309311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367d899-117d-4a4b-9cdf-285641c986ab None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.309770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367d899-117d-4a4b-9cdf-285641c986ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.331064 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1367d899-117d-4a4b-9cdf-285641c986ab None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:59.333139 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1367d899-117d-4a4b-9cdf-285641c986ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:59.408536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e1ef6-7a7e-4f85-b68c-700e53d30e83 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:59.415323 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 462/931] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:38:59.416528 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367d899-117d-4a4b-9cdf-285641c986ab None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:59.430163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:59.460320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510d1d1-c42c-48fa-9a34-2da36e0edaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:59.462189 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510d1d1-c42c-48fa-9a34-2da36e0edaab None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.462629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510d1d1-c42c-48fa-9a34-2da36e0edaab None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.463084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510d1d1-c42c-48fa-9a34-2da36e0edaab None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.463929 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510d1d1-c42c-48fa-9a34-2da36e0edaab None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:59.464388 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510d1d1-c42c-48fa-9a34-2da36e0edaab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:59.466000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510d1d1-c42c-48fa-9a34-2da36e0edaab None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:59.489943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 470/932] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:59.495068 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510d1d1-c42c-48fa-9a34-2da36e0edaab None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:59.556738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367d899-117d-4a4b-9cdf-285641c986ab None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:59.600980 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:59.604386 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ccd20-74da-449f-be65-7a1060315a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:59.605087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ccd20-74da-449f-be65-7a1060315a4a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.605557 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ccd20-74da-449f-be65-7a1060315a4a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.605828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ccd20-74da-449f-be65-7a1060315a4a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.606807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ccd20-74da-449f-be65-7a1060315a4a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:59.608502 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ccd20-74da-449f-be65-7a1060315a4a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-a93ccd20-74da-449f-be65-7a1060315a4a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:59.622598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:59.623762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:59.623762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:59.623762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:59.623762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:59.623762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:59.623762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:59.623762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:38:59.623762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:38:59.623762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:38:59.623762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:38:59.623762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:59.623762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:59.623762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:59.623762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:59.623762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:59.623762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:59.624678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:59.624678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:59.624678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:59.624678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:59.624678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:59.624678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:59.624678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:59.624678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:59.624678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:59.624678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:59.624678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:38:59.624678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:59.624678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:38:59.624678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:38:59.624678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 463/933] 10.222.0.91 () {56 vars in 1126 bytes} [Thu Sep 8 13:38:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:59.630120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ccd20-74da-449f-be65-7a1060315a4a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:59.630358 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510d1d1-c42c-48fa-9a34-2da36e0edaab None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:59.683402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6691640-2bd5-4b48-8dc3-28a828e154e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:59.684281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6691640-2bd5-4b48-8dc3-28a828e154e7 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.685033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6691640-2bd5-4b48-8dc3-28a828e154e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.685309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6691640-2bd5-4b48-8dc3-28a828e154e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.694134 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6691640-2bd5-4b48-8dc3-28a828e154e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:59.695960 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6691640-2bd5-4b48-8dc3-28a828e154e7 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:59.708424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-e6691640-2bd5-4b48-8dc3-28a828e154e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:59.708424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:59.708424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:59.708424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:59.708424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:59.708424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:59.708424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:59.708424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:59.708424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:59.708424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:59.708424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:59.708424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:59.708424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:59.708424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:59.708424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:59.709340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:59.710295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:59.710295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:59.710295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:59.710295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:59.710295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:59.710295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:59.710295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:59.710295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:59.710295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 471/934] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:59.710295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6691640-2bd5-4b48-8dc3-28a828e154e7 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:59.718158 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:59.757538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ccd20-74da-449f-be65-7a1060315a4a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:59.793419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f78a24-ec07-4f53-b394-a376fefbb4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:59.794015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f78a24-ec07-4f53-b394-a376fefbb4c2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.794298 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f78a24-ec07-4f53-b394-a376fefbb4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.794551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f78a24-ec07-4f53-b394-a376fefbb4c2 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.795218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f78a24-ec07-4f53-b394-a376fefbb4c2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:59.795572 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f78a24-ec07-4f53-b394-a376fefbb4c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:59.797449 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f78a24-ec07-4f53-b394-a376fefbb4c2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:59.803125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 464/935] 10.222.0.91 () {56 vars in 1122 bytes} [Thu Sep 8 13:38:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:59.808367 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f78a24-ec07-4f53-b394-a376fefbb4c2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:59.818010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6691640-2bd5-4b48-8dc3-28a828e154e7 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:59.852176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5127820-8571-467e-9a10-2d96291e9157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:59.852802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5127820-8571-467e-9a10-2d96291e9157 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.853076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5127820-8571-467e-9a10-2d96291e9157 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.853344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5127820-8571-467e-9a10-2d96291e9157 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.853916 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5127820-8571-467e-9a10-2d96291e9157 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:59.855325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5127820-8571-467e-9a10-2d96291e9157 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:59.858134 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:59.865196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-b5127820-8571-467e-9a10-2d96291e9157 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:38:59.865196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:59.865196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:38:59.865196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:38:59.865196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:38:59.865196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:38:59.865196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:38:59.865196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:59.865196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:38:59.865196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:59.865196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:59.865196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:59.865196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:59.865196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:59.865196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:59.865196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:59.873338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:59.874215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:59.874215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:59.874215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:59.874215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:59.874215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:59.874215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:59.874215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:59.874215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:59.874215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:59.874215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:59.874215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:38:59.874215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:59.874215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:38:59.874215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:38:59.874215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 472/936] 10.222.0.91 () {56 vars in 1132 bytes} [Thu Sep 8 13:38:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:59.875054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5127820-8571-467e-9a10-2d96291e9157 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:59.933430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f78a24-ec07-4f53-b394-a376fefbb4c2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:59.976134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c4bcf-a72c-4443-8278-9ecd29451555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:59.976852 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c4bcf-a72c-4443-8278-9ecd29451555 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.977232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c4bcf-a72c-4443-8278-9ecd29451555 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.977594 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c4bcf-a72c-4443-8278-9ecd29451555 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.978296 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c4bcf-a72c-4443-8278-9ecd29451555 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:59.978598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c4bcf-a72c-4443-8278-9ecd29451555 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:59.980373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c4bcf-a72c-4443-8278-9ecd29451555 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:59.986200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:00.005731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 465/937] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:00.005731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c4bcf-a72c-4443-8278-9ecd29451555 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.006137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5127820-8571-467e-9a10-2d96291e9157 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.044433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7a74b-495d-427a-81bc-e2f0a6f23ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.045033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7a74b-495d-427a-81bc-e2f0a6f23ac2 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.045364 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7a74b-495d-427a-81bc-e2f0a6f23ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.045640 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7a74b-495d-427a-81bc-e2f0a6f23ac2 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.046265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c7a74b-495d-427a-81bc-e2f0a6f23ac2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:00.046564 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c7a74b-495d-427a-81bc-e2f0a6f23ac2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:00.047974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c7a74b-495d-427a-81bc-e2f0a6f23ac2 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:00.054367 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 473/938] 10.222.0.91 () {56 vars in 1122 bytes} [Thu Sep 8 13:38:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:00.055185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7a74b-495d-427a-81bc-e2f0a6f23ac2 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.120484 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c4bcf-a72c-4443-8278-9ecd29451555 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.128837 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:00.159973 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10091d-6799-4c49-acea-41300a2b0117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.160521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10091d-6799-4c49-acea-41300a2b0117 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.160789 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10091d-6799-4c49-acea-41300a2b0117 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.161053 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10091d-6799-4c49-acea-41300a2b0117 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.161688 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10091d-6799-4c49-acea-41300a2b0117 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:00.161977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10091d-6799-4c49-acea-41300a2b0117 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:00.163393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f10091d-6799-4c49-acea-41300a2b0117 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:00.169240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 466/939] 10.222.0.91 () {56 vars in 1126 bytes} [Thu Sep 8 13:39:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:00.170007 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10091d-6799-4c49-acea-41300a2b0117 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.177545 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7a74b-495d-427a-81bc-e2f0a6f23ac2 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.229561 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbf083-85e3-4923-bc62-d76fdfee87c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.230719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbf083-85e3-4923-bc62-d76fdfee87c9 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.231130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbf083-85e3-4923-bc62-d76fdfee87c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.231503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbf083-85e3-4923-bc62-d76fdfee87c9 None admin] PATH_INFO: `/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/46cce59255af494eaa238ab8c8cddc51/roles/cf5de1e5771c4b60911d58b68de13600` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.235115 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbf083-85e3-4923-bc62-d76fdfee87c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb2bd53d40841eeb1653bd5927fb189, user_id=46cce59255af494eaa238ab8c8cddc51, role_id=cf5de1e5771c4b60911d58b68de13600)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:00.236944 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbf083-85e3-4923-bc62-d76fdfee87c9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:00.251827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:00.299459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10091d-6799-4c49-acea-41300a2b0117 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.337086 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 474/940] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:39:00 2022] PUT /identity/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/46cce59255af494eaa238ab8c8cddc51/roles/cf5de1e5771c4b60911d58b68de13600 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:39:00.338876 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbf083-85e3-4923-bc62-d76fdfee87c9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.349047 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376fb42-ea37-4bdf-a321-08ec9b84d37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.349961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376fb42-ea37-4bdf-a321-08ec9b84d37b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.350420 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376fb42-ea37-4bdf-a321-08ec9b84d37b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.350681 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376fb42-ea37-4bdf-a321-08ec9b84d37b None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.351472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376fb42-ea37-4bdf-a321-08ec9b84d37b None admin] RBAC: Adding query filter params (interface=public, service_id=5b87c5314d264435b9dfea5b1777d1b2, region_id=RegionOne) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:00.351782 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376fb42-ea37-4bdf-a321-08ec9b84d37b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:00.353217 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376fb42-ea37-4bdf-a321-08ec9b84d37b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:00.363634 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 467/941] 10.222.0.91 () {56 vars in 1251 bytes} [Thu Sep 8 13:39:00 2022] GET /identity/v3/endpoints?service_id=5b87c5314d264435b9dfea5b1777d1b2&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:00.368532 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376fb42-ea37-4bdf-a321-08ec9b84d37b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.505809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbf083-85e3-4923-bc62-d76fdfee87c9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.514130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376fb42-ea37-4bdf-a321-08ec9b84d37b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.542376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:00.545971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:00.560593 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068e4bb-b62d-449a-803d-38619fb9c433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.561343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e4bb-b62d-449a-803d-38619fb9c433 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.561592 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e4bb-b62d-449a-803d-38619fb9c433 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.561860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068e4bb-b62d-449a-803d-38619fb9c433 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.562473 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068e4bb-b62d-449a-803d-38619fb9c433 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:00.564233 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068e4bb-b62d-449a-803d-38619fb9c433 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:00.564491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac74ab1-5d7c-4134-b3a9-c43f219d0dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.565111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac74ab1-5d7c-4134-b3a9-c43f219d0dc6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.565606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac74ab1-5d7c-4134-b3a9-c43f219d0dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.565913 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac74ab1-5d7c-4134-b3a9-c43f219d0dc6 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.566912 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac74ab1-5d7c-4134-b3a9-c43f219d0dc6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:00.567221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac74ab1-5d7c-4134-b3a9-c43f219d0dc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:00.568934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac74ab1-5d7c-4134-b3a9-c43f219d0dc6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:00.571807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 468/942] 10.222.0.91 () {56 vars in 1087 bytes} [Thu Sep 8 13:39:00 2022] GET /identity/v3/services => generated 1497 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:39:00.578663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 475/943] 10.222.0.91 () {56 vars in 1126 bytes} [Thu Sep 8 13:39:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:00.586124 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068e4bb-b62d-449a-803d-38619fb9c433 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.723032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068e4bb-b62d-449a-803d-38619fb9c433 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.764061 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee93d7-0803-4dbe-873d-c231020db92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.764705 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee93d7-0803-4dbe-873d-c231020db92b None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.765168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee93d7-0803-4dbe-873d-c231020db92b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.765638 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee93d7-0803-4dbe-873d-c231020db92b None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.767367 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee93d7-0803-4dbe-873d-c231020db92b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:00.768767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee93d7-0803-4dbe-873d-c231020db92b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:00.859277 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 469/944] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:39:00 2022] POST /identity/v3/endpoints => generated 347 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:01.627607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34654a05-1266-45e9-8f2b-c879fc5e4cc6 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.628171 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34654a05-1266-45e9-8f2b-c879fc5e4cc6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.628171 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34654a05-1266-45e9-8f2b-c879fc5e4cc6 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.629880 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 476/945] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:01.635680 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9871e82-8bb8-438d-9463-3e856f9fea37 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.636260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9871e82-8bb8-438d-9463-3e856f9fea37 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.636351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9871e82-8bb8-438d-9463-3e856f9fea37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.656558 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b9871e82-8bb8-438d-9463-3e856f9fea37 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:01.657778 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b9871e82-8bb8-438d-9463-3e856f9fea37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:01.706132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:01.744593 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 470/946] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:02.020737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34654a05-1266-45e9-8f2b-c879fc5e4cc6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:02.133656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34654a05-1266-45e9-8f2b-c879fc5e4cc6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:02.168805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b093d2-633b-4b11-b931-221476e34cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:02.169810 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b093d2-633b-4b11-b931-221476e34cc0 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:02.170315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b093d2-633b-4b11-b931-221476e34cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:02.170836 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b093d2-633b-4b11-b931-221476e34cc0 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:02.171716 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b093d2-633b-4b11-b931-221476e34cc0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:02.173327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b093d2-633b-4b11-b931-221476e34cc0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:02.176751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: WARNING keystone.server.flask.common [None req-e7b093d2-633b-4b11-b931-221476e34cc0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 08 13:39:02.234160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.sql.core [None req-e7b093d2-633b-4b11-b931-221476e34cc0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 13:39:02.234160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 08 13:39:02.234160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [parameters: {'id': '77202edfbad7432199dc83a6b0146b5e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 08 13:39:02.234160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93172) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 13:39:02.251941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-e7b093d2-633b-4b11-b931-221476e34cc0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 77202edfbad7432199dc83a6b0146b5e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 77202edfbad7432199dc83a6b0146b5e. Sep 08 13:39:02.251941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:02.251941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:39:02.251941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:39:02.251941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:39:02.251941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:39:02.251941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:39:02.251941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:39:02.251941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:39:02.251941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:39:02.251941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:39:02.251941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:39:02.251941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application result.read() Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 08 13:39:02.252806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application yield resource Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise self.value Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application session.commit() Sep 08 13:39:02.253760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self.session.flush() Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application compat.raise_( Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 13:39:02.254620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 13:39:02.256305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 13:39:02.256305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 13:39:02.256305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 13:39:02.256305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 13:39:02.256305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 08 13:39:02.256305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 08 13:39:02.256305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 13:39:02.256305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 13:39:02.256305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 13:39:02.256305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 13:39:02.256305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 13:39:02.256305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 13:39:02.256305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 13:39:02.256305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 13:39:02.256305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application result.read() Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:39:02.258018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:39:02.260197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:39:02.260197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:39:02.260197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:39:02.260197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:39:02.260197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:39:02.260197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:39:02.260197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 13:39:02.260197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 08 13:39:02.260197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [parameters: {'id': '77202edfbad7432199dc83a6b0146b5e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 08 13:39:02.260197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 13:39:02.260197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:02.260197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:02.260197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:02.260197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:02.260197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:02.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:02.264264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:02.264264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 08 13:39:02.264264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 08 13:39:02.264264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:02.264264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:02.264264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 08 13:39:02.264264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 08 13:39:02.264264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 77202edfbad7432199dc83a6b0146b5e. Sep 08 13:39:02.264264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:02.267048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 477/947] 10.222.0.91 () {60 vars in 1140 bytes} [Thu Sep 8 13:39:02 2022] POST /identity/v3/projects => generated 262 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:02.273064 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9871e82-8bb8-438d-9463-3e856f9fea37 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:02.380915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9871e82-8bb8-438d-9463-3e856f9fea37 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:02.419232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e798a4-8351-4ff1-a8d6-9d950d4b68e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:02.419920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e798a4-8351-4ff1-a8d6-9d950d4b68e3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:02.420202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e798a4-8351-4ff1-a8d6-9d950d4b68e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:02.420475 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e798a4-8351-4ff1-a8d6-9d950d4b68e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:02.428150 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e798a4-8351-4ff1-a8d6-9d950d4b68e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:02.429681 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e798a4-8351-4ff1-a8d6-9d950d4b68e3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:02.443446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-a8e798a4-8351-4ff1-a8d6-9d950d4b68e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:02.443446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:02.443446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:02.443446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:02.443446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:02.443446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:02.443446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:02.443446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:02.443446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:02.443446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:02.443446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:02.443446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:02.443446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:02.443446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:02.443446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:02.445808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:02.447933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:02.447933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:02.447933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:02.447933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:02.447933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:02.447933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:02.447933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:02.447933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:02.447933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 471/948] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:02.451644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b093d2-633b-4b11-b931-221476e34cc0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:02.510501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:02.573886 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b093d2-633b-4b11-b931-221476e34cc0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:02.616039 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b763ca0-2990-4dc3-8d6f-7b71c3d91a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:02.616667 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b763ca0-2990-4dc3-8d6f-7b71c3d91a40 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:02.616944 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b763ca0-2990-4dc3-8d6f-7b71c3d91a40 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:02.617215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b763ca0-2990-4dc3-8d6f-7b71c3d91a40 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:02.617886 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b763ca0-2990-4dc3-8d6f-7b71c3d91a40 None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:02.618187 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b763ca0-2990-4dc3-8d6f-7b71c3d91a40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:02.619889 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b763ca0-2990-4dc3-8d6f-7b71c3d91a40 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:02.641864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 478/949] 10.222.0.91 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:02 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:02.658239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b44809-3ac4-4ebd-8936-b2b4c76895c0 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:02.658758 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b44809-3ac4-4ebd-8936-b2b4c76895c0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:02.658758 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b44809-3ac4-4ebd-8936-b2b4c76895c0 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:02.660547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 472/950] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:02.666452 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844a022-8aa1-432b-9e9b-0c9170a0bb43 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:02.666753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844a022-8aa1-432b-9e9b-0c9170a0bb43 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:02.667074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844a022-8aa1-432b-9e9b-0c9170a0bb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:02.688887 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b844a022-8aa1-432b-9e9b-0c9170a0bb43 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:02.690596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b844a022-8aa1-432b-9e9b-0c9170a0bb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:02.770947 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 479/951] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:02.915355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ee5c7-8651-488d-99e9-c84e37598d7c None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:02.915641 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ee5c7-8651-488d-99e9-c84e37598d7c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:02.915920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ee5c7-8651-488d-99e9-c84e37598d7c None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:02.917363 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 473/952] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:02.923159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d1cf1-04f8-4703-954b-826f96eb7d9c None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:02.923449 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d1cf1-04f8-4703-954b-826f96eb7d9c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:02.923725 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d1cf1-04f8-4703-954b-826f96eb7d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:02.946507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-7f2d1cf1-04f8-4703-954b-826f96eb7d9c None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:02.947782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7f2d1cf1-04f8-4703-954b-826f96eb7d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:02.981561 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.024476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 480/953] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:03.057068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ee5c7-8651-488d-99e9-c84e37598d7c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.090135 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.173069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ee5c7-8651-488d-99e9-c84e37598d7c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.209756 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa5d41-2fa7-4109-8b6c-16d13675455f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.210398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa5d41-2fa7-4109-8b6c-16d13675455f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.210678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa5d41-2fa7-4109-8b6c-16d13675455f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.210972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa5d41-2fa7-4109-8b6c-16d13675455f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.221812 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa5d41-2fa7-4109-8b6c-16d13675455f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:03.222743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017ac83-991a-4c75-8b25-e3a87884fde0 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.223144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017ac83-991a-4c75-8b25-e3a87884fde0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.223349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017ac83-991a-4c75-8b25-e3a87884fde0 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.224946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 481/954] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:03.226797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-d1fa5d41-2fa7-4109-8b6c-16d13675455f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:03.226797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:03.226797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:03.226797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:03.226797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:03.226797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:03.226797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:03.226797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:03.226797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:03.226797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:03.226797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:03.226797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:03.226797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:03.226797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:03.226797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:03.226797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:03.227771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:03.227771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:03.227771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:03.227771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:03.227771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:03.227771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:03.227771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:03.227771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:03.227771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:03.227771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:03.227771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:03.227771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:03.227771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:03.227771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:03.227771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:03.227771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:03.229956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:03.229956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:03.229956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:03.229956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:03.229956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:03.229956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:03.229956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:03.229956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:03.229956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:03.229956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:03.229956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:03.229956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:03.229956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 474/955] 10.222.0.91 () {56 vars in 1120 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:03.231644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1204c-2e35-4062-9e8b-1de7ae1b3d1f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.231644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1204c-2e35-4062-9e8b-1de7ae1b3d1f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.231644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1204c-2e35-4062-9e8b-1de7ae1b3d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.234221 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa5d41-2fa7-4109-8b6c-16d13675455f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.249910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-85e1204c-2e35-4062-9e8b-1de7ae1b3d1f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:03.251292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-85e1204c-2e35-4062-9e8b-1de7ae1b3d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:03.332204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 482/956] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:03.336150 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1204c-2e35-4062-9e8b-1de7ae1b3d1f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.341590 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.355782 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa5d41-2fa7-4109-8b6c-16d13675455f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.396930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afeb4d-229b-4ef7-9325-aeaf5ecd8b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.397533 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afeb4d-229b-4ef7-9325-aeaf5ecd8b10 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.397812 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afeb4d-229b-4ef7-9325-aeaf5ecd8b10 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.398120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afeb4d-229b-4ef7-9325-aeaf5ecd8b10 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.398837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afeb4d-229b-4ef7-9325-aeaf5ecd8b10 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:03.399193 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afeb4d-229b-4ef7-9325-aeaf5ecd8b10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:03.400914 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afeb4d-229b-4ef7-9325-aeaf5ecd8b10 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:03.410097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 475/957] 10.222.0.91 () {56 vars in 1115 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:03.415908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afeb4d-229b-4ef7-9325-aeaf5ecd8b10 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.474930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.486172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1204c-2e35-4062-9e8b-1de7ae1b3d1f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.530330 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8306fb-76cd-444c-8803-4301c905c879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.530986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8306fb-76cd-444c-8803-4301c905c879 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.531265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8306fb-76cd-444c-8803-4301c905c879 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.531531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8306fb-76cd-444c-8803-4301c905c879 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.532110 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8306fb-76cd-444c-8803-4301c905c879 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:03.534024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8306fb-76cd-444c-8803-4301c905c879 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-4a8306fb-76cd-444c-8803-4301c905c879 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:03.546970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:03.548351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:03.548351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:03.548351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:03.548351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:03.548351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:03.548351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:03.548351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:03.548351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:03.548351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:03.548351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:03.548351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:03.548351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:03.548351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:03.548351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:03.548351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:03.548351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:03.549329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:03.549329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:03.549329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:03.549329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:03.549329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:03.549329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:03.549329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:03.549329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:03.549329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:03.549329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:03.549329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:03.549329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:03.549329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:03.549329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:03.550607 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afeb4d-229b-4ef7-9325-aeaf5ecd8b10 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.553573 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 483/958] 10.222.0.91 () {56 vars in 1126 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:03.555753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8306fb-76cd-444c-8803-4301c905c879 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.591392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97140ee2-2481-4341-8203-30e59910f12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.592303 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97140ee2-2481-4341-8203-30e59910f12c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.592800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97140ee2-2481-4341-8203-30e59910f12c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.593220 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97140ee2-2481-4341-8203-30e59910f12c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.606397 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97140ee2-2481-4341-8203-30e59910f12c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:03.607886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97140ee2-2481-4341-8203-30e59910f12c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:03.622625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-97140ee2-2481-4341-8203-30e59910f12c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:03.622625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:03.622625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:03.622625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:03.622625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:03.622625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:03.622625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:03.622625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:03.622625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:03.622625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:03.622625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:03.622625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:03.622625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:03.622625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:03.622625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:03.622625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:03.629042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:03.630220 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:03.630220 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:03.630220 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:03.630220 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 476/959] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:03.630220 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97140ee2-2481-4341-8203-30e59910f12c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.632567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.686437 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8306fb-76cd-444c-8803-4301c905c879 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.724629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665acdf-095c-4b7d-bbd7-a558227291f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.725301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665acdf-095c-4b7d-bbd7-a558227291f8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.725501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665acdf-095c-4b7d-bbd7-a558227291f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.725778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665acdf-095c-4b7d-bbd7-a558227291f8 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.726413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665acdf-095c-4b7d-bbd7-a558227291f8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:03.726712 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665acdf-095c-4b7d-bbd7-a558227291f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:03.728092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665acdf-095c-4b7d-bbd7-a558227291f8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:03.737590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 484/960] 10.222.0.91 () {56 vars in 1122 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:03.737590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665acdf-095c-4b7d-bbd7-a558227291f8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.748822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97140ee2-2481-4341-8203-30e59910f12c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.766943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.786570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281ee88-82ac-4bb2-952d-ab43b0ab3ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.787215 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281ee88-82ac-4bb2-952d-ab43b0ab3ca3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.787488 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281ee88-82ac-4bb2-952d-ab43b0ab3ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.787762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281ee88-82ac-4bb2-952d-ab43b0ab3ca3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.788942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281ee88-82ac-4bb2-952d-ab43b0ab3ca3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:03.789677 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281ee88-82ac-4bb2-952d-ab43b0ab3ca3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-4281ee88-82ac-4bb2-952d-ab43b0ab3ca3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:03.799467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:03.800517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:03.800517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:03.800517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:03.800517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:03.800517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:03.800517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:03.800517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:03.800517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:03.800517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:03.800517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:03.800517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:03.800517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:03.800517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:03.800517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:03.800517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:03.800517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:03.801378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:03.801378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:03.801378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:03.801378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:03.801378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:03.801378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:03.801378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:03.801378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:03.801378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:03.801378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:03.801378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:03.801378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:03.801378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:03.801378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:03.801378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 477/961] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:03.802189 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281ee88-82ac-4bb2-952d-ab43b0ab3ca3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.852629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665acdf-095c-4b7d-bbd7-a558227291f8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.879381 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.888215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a26095-b8c9-4a46-a7ab-552ccd8973f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.893073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a26095-b8c9-4a46-a7ab-552ccd8973f6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.893073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a26095-b8c9-4a46-a7ab-552ccd8973f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.893073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a26095-b8c9-4a46-a7ab-552ccd8973f6 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.893073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a26095-b8c9-4a46-a7ab-552ccd8973f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:03.893073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a26095-b8c9-4a46-a7ab-552ccd8973f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:03.893073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a26095-b8c9-4a46-a7ab-552ccd8973f6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:03.908924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 485/962] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:03.908924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a26095-b8c9-4a46-a7ab-552ccd8973f6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.920515 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281ee88-82ac-4bb2-952d-ab43b0ab3ca3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.956260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731338b4-4ab7-4491-85c3-57935c113e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.956889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731338b4-4ab7-4491-85c3-57935c113e7e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.957162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731338b4-4ab7-4491-85c3-57935c113e7e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.957457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731338b4-4ab7-4491-85c3-57935c113e7e None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.958048 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731338b4-4ab7-4491-85c3-57935c113e7e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:03.958610 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731338b4-4ab7-4491-85c3-57935c113e7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:03.960013 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731338b4-4ab7-4491-85c3-57935c113e7e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:03.969640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 478/963] 10.222.0.91 () {56 vars in 1122 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:03.969640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731338b4-4ab7-4491-85c3-57935c113e7e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.995533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:04.014257 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a26095-b8c9-4a46-a7ab-552ccd8973f6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.055391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f3494-6b16-4c4f-be38-159a457a4d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.057025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f3494-6b16-4c4f-be38-159a457a4d0d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.057547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f3494-6b16-4c4f-be38-159a457a4d0d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.058101 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f3494-6b16-4c4f-be38-159a457a4d0d None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.059094 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f3494-6b16-4c4f-be38-159a457a4d0d None admin] RBAC: Adding query filter params (name=compute) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:04.059566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f3494-6b16-4c4f-be38-159a457a4d0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.061123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f3494-6b16-4c4f-be38-159a457a4d0d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.066942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 486/964] 10.222.0.91 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:04.067907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f3494-6b16-4c4f-be38-159a457a4d0d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.084995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731338b4-4ab7-4491-85c3-57935c113e7e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.116221 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:04.121050 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a61857-de9e-460f-b687-d107f42e3b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.121665 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a61857-de9e-460f-b687-d107f42e3b80 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.121937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a61857-de9e-460f-b687-d107f42e3b80 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.122206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a61857-de9e-460f-b687-d107f42e3b80 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.130333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a61857-de9e-460f-b687-d107f42e3b80 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.131787 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a61857-de9e-460f-b687-d107f42e3b80 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.142633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-79a61857-de9e-460f-b687-d107f42e3b80 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:39:04.142633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:04.142633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:04.142633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:04.142633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:04.142633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:04.142633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:04.142633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:04.142633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:04.142633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:04.142633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:04.142633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:04.142633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:04.142633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:04.142633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:04.142633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:04.143805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:04.144779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:04.144779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:04.144779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:04.144779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:04.144779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:04.144779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:04.144779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:04.144779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:04.144779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:39:04.144779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:04.144779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 479/965] 10.222.0.91 () {56 vars in 1117 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:04.145464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a61857-de9e-460f-b687-d107f42e3b80 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.180815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f3494-6b16-4c4f-be38-159a457a4d0d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.217103 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ab6f4-2583-4e66-a974-96b505d937a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.217711 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ab6f4-2583-4e66-a974-96b505d937a7 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.217993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ab6f4-2583-4e66-a974-96b505d937a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.218265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ab6f4-2583-4e66-a974-96b505d937a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.218947 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ab6f4-2583-4e66-a974-96b505d937a7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.220327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ab6f4-2583-4e66-a974-96b505d937a7 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.256817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a61857-de9e-460f-b687-d107f42e3b80 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.289448 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c2566-271c-4a6d-9515-2372c7b26ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.290066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c2566-271c-4a6d-9515-2372c7b26ddf None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.290355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c2566-271c-4a6d-9515-2372c7b26ddf None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.290620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c2566-271c-4a6d-9515-2372c7b26ddf None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.291298 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c2566-271c-4a6d-9515-2372c7b26ddf None admin] RBAC: Adding query filter params (type=compute) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:04.291604 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c2566-271c-4a6d-9515-2372c7b26ddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.293051 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0c2566-271c-4a6d-9515-2372c7b26ddf None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.297472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 480/966] 10.222.0.91 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:04 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:04.298245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c2566-271c-4a6d-9515-2372c7b26ddf None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.350918 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 487/967] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:39:04 2022] POST /identity/v3/endpoints => generated 349 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:04.388424 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:04.450967 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c2566-271c-4a6d-9515-2372c7b26ddf None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.507177 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db56ca1-7eaf-49d4-be57-1db1361efd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.507679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db56ca1-7eaf-49d4-be57-1db1361efd8a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.507960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db56ca1-7eaf-49d4-be57-1db1361efd8a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.508230 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db56ca1-7eaf-49d4-be57-1db1361efd8a None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.508979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db56ca1-7eaf-49d4-be57-1db1361efd8a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:04.509286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db56ca1-7eaf-49d4-be57-1db1361efd8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.510766 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db56ca1-7eaf-49d4-be57-1db1361efd8a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.532086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 481/968] 10.222.0.91 () {56 vars in 1149 bytes} [Thu Sep 8 13:39:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:04.539005 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ab6f4-2583-4e66-a974-96b505d937a7 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.545988 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:04.654562 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ab6f4-2583-4e66-a974-96b505d937a7 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.691741 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8585f0f-8814-4de3-ac6e-8f3a277a7793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.692404 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8585f0f-8814-4de3-ac6e-8f3a277a7793 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.692662 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8585f0f-8814-4de3-ac6e-8f3a277a7793 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.692937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8585f0f-8814-4de3-ac6e-8f3a277a7793 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.766383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8585f0f-8814-4de3-ac6e-8f3a277a7793 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.767798 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8585f0f-8814-4de3-ac6e-8f3a277a7793 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.769737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:04.874616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 488/969] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:04 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 13:39:04.880141 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db56ca1-7eaf-49d4-be57-1db1361efd8a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.990396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db56ca1-7eaf-49d4-be57-1db1361efd8a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:05.001822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:05.013493 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a0688-ccc3-4935-b724-f002de8aefce None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.014344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a0688-ccc3-4935-b724-f002de8aefce None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.014344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a0688-ccc3-4935-b724-f002de8aefce None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.017213 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 489/970] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:05.021466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0137c6a-9994-4f2e-a884-4ccb5ebb323c None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.021861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0137c6a-9994-4f2e-a884-4ccb5ebb323c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.022241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0137c6a-9994-4f2e-a884-4ccb5ebb323c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.028143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02fa25-ec37-493c-ad59-3df0517bf403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:05.029004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02fa25-ec37-493c-ad59-3df0517bf403 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.029433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02fa25-ec37-493c-ad59-3df0517bf403 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.029882 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02fa25-ec37-493c-ad59-3df0517bf403 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.041332 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02fa25-ec37-493c-ad59-3df0517bf403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:05.041517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b0137c6a-9994-4f2e-a884-4ccb5ebb323c None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:05.042823 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b0137c6a-9994-4f2e-a884-4ccb5ebb323c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:05.043317 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02fa25-ec37-493c-ad59-3df0517bf403 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:05.063690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-fc02fa25-ec37-493c-ad59-3df0517bf403 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:05.063690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:05.063690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:05.063690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:05.063690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:05.063690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:05.063690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:05.063690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:05.063690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:05.063690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:05.063690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:05.063690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:05.063690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:05.063690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:05.063690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:05.063690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:05.064915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:05.066755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:05.066755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:05.066755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:05.069400 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 482/971] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:05.075434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02fa25-ec37-493c-ad59-3df0517bf403 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:05.123037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 490/972] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:05.186989 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02fa25-ec37-493c-ad59-3df0517bf403 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:05.223222 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486545b1-1d60-471d-8d73-7ea1657c75a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:05.224109 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486545b1-1d60-471d-8d73-7ea1657c75a2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.224538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486545b1-1d60-471d-8d73-7ea1657c75a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.224961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486545b1-1d60-471d-8d73-7ea1657c75a2 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.225808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486545b1-1d60-471d-8d73-7ea1657c75a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:05.226272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486545b1-1d60-471d-8d73-7ea1657c75a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:05.227923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486545b1-1d60-471d-8d73-7ea1657c75a2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:05.242634 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 483/973] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:05.252044 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0137c6a-9994-4f2e-a884-4ccb5ebb323c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:05.288216 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:05.362604 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0137c6a-9994-4f2e-a884-4ccb5ebb323c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:05.400618 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38032605-1f59-459c-92c2-5f0df1a87ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:05.401272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38032605-1f59-459c-92c2-5f0df1a87ab2 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.401550 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38032605-1f59-459c-92c2-5f0df1a87ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.401803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38032605-1f59-459c-92c2-5f0df1a87ab2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.409373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38032605-1f59-459c-92c2-5f0df1a87ab2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:05.410964 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38032605-1f59-459c-92c2-5f0df1a87ab2 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:05.417154 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486545b1-1d60-471d-8d73-7ea1657c75a2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:05.425737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-38032605-1f59-459c-92c2-5f0df1a87ab2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:05.425737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:05.425737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:05.425737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:05.425737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:05.425737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:05.425737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:05.425737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:05.425737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:05.425737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:05.425737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:05.425737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:05.425737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:05.425737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:05.425737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:05.426699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:05.427663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:05.427663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:05.427663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:05.427663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:05.427663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:05.427663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:05.427663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:05.427663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:05.431024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 491/974] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:05.433983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38032605-1f59-459c-92c2-5f0df1a87ab2 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:05.471266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:05.546187 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486545b1-1d60-471d-8d73-7ea1657c75a2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:05.560778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:05.571399 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38032605-1f59-459c-92c2-5f0df1a87ab2 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:05.596743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022a84b-06b9-4a33-8f51-7723308a7bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:05.598461 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022a84b-06b9-4a33-8f51-7723308a7bbb None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.598947 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022a84b-06b9-4a33-8f51-7723308a7bbb None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.599428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022a84b-06b9-4a33-8f51-7723308a7bbb None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.600426 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022a84b-06b9-4a33-8f51-7723308a7bbb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:05.602671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022a84b-06b9-4a33-8f51-7723308a7bbb None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:05.606654 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: WARNING keystone.server.flask.common [None req-5022a84b-06b9-4a33-8f51-7723308a7bbb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 08 13:39:05.608905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d575af1-064a-4e22-a328-6d2fce51d09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:05.609589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d575af1-064a-4e22-a328-6d2fce51d09a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.609935 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d575af1-064a-4e22-a328-6d2fce51d09a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.610280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d575af1-064a-4e22-a328-6d2fce51d09a None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.611075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d575af1-064a-4e22-a328-6d2fce51d09a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:05.611450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d575af1-064a-4e22-a328-6d2fce51d09a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:05.613606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d575af1-064a-4e22-a328-6d2fce51d09a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:05.626631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 492/975] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:05.632485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d575af1-064a-4e22-a328-6d2fce51d09a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:05.668032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.sql.core [None req-5022a84b-06b9-4a33-8f51-7723308a7bbb None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 13:39:05.668032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 08 13:39:05.668032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [parameters: {'user_id': 'c523106e3dd541e1b1c33a438743779b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 08 13:39:05.668032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93173) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 13:39:05.676942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-5022a84b-06b9-4a33-8f51-7723308a7bbb None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 08 13:39:05.676942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:05.676942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:39:05.676942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:39:05.676942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:39:05.676942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:39:05.676942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:39:05.676942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:39:05.676942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:39:05.676942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:39:05.676942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:39:05.676942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:39:05.676942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:39:05.676942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result.read() Sep 08 13:39:05.676942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:39:05.677816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application yield resource Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise self.value Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application session.commit() Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 13:39:05.678762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.session.flush() Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application compat.raise_( Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 13:39:05.679764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 13:39:05.680655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 13:39:05.680655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 13:39:05.680655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 08 13:39:05.680655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 08 13:39:05.680655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 13:39:05.680655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 13:39:05.680655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 13:39:05.680655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 13:39:05.680655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 13:39:05.680655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 13:39:05.680655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 13:39:05.680655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 13:39:05.680655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 13:39:05.680655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 13:39:05.680655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:39:05.680655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception Sep 08 13:39:05.681503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:39:05.681503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:39:05.681503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:39:05.681503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:39:05.681503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:39:05.681503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:39:05.681503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:39:05.681503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:39:05.681503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:39:05.681503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:39:05.681503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:39:05.681503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result.read() Sep 08 13:39:05.681503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:39:05.681503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:39:05.681503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:39:05.681503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:39:05.683970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:39:05.683970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:39:05.683970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:39:05.683970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:39:05.683970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 13:39:05.683970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 08 13:39:05.683970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c523106e3dd541e1b1c33a438743779b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 08 13:39:05.683970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 13:39:05.683970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:05.683970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:05.683970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:05.683970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:05.683970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:05.683970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:05.683970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 08 13:39:05.686021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 08 13:39:05.688207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 08 13:39:05.688207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 08 13:39:05.688207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 13:39:05.688207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 13:39:05.688207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 08 13:39:05.688207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:05.688207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 484/976] 10.222.0.91 () {60 vars in 1131 bytes} [Thu Sep 8 13:39:05 2022] POST /identity/v3/users => generated 160 bytes in 262 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:05.688207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022a84b-06b9-4a33-8f51-7723308a7bbb None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:05.781057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d575af1-064a-4e22-a328-6d2fce51d09a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:05.783670 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:05.802319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022a84b-06b9-4a33-8f51-7723308a7bbb None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:05.827913 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800da8c-86c3-4219-9aac-82fd7179b710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:05.830481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800da8c-86c3-4219-9aac-82fd7179b710 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.831004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800da8c-86c3-4219-9aac-82fd7179b710 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.832265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800da8c-86c3-4219-9aac-82fd7179b710 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.834090 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800da8c-86c3-4219-9aac-82fd7179b710 None admin] RBAC: Adding query filter params (role.id=cf5de1e5771c4b60911d58b68de13600, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=46cce59255af494eaa238ab8c8cddc51) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:05.834608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800da8c-86c3-4219-9aac-82fd7179b710 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:05.837176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800da8c-86c3-4219-9aac-82fd7179b710 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:05.841495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0b484-8f01-4dd4-9d00-8b25cb84d617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:05.847370 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0b484-8f01-4dd4-9d00-8b25cb84d617 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.847370 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0b484-8f01-4dd4-9d00-8b25cb84d617 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.847370 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0b484-8f01-4dd4-9d00-8b25cb84d617 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.848163 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 493/977] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:05 2022] GET /identity/v3/role_assignments?user.id=46cce59255af494eaa238ab8c8cddc51&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=cf5de1e5771c4b60911d58b68de13600 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:05.859556 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0b484-8f01-4dd4-9d00-8b25cb84d617 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:05.861707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0b484-8f01-4dd4-9d00-8b25cb84d617 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:05.875237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-1fb0b484-8f01-4dd4-9d00-8b25cb84d617 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:39:05.875237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:05.875237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:05.875237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:05.875237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:05.875237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:05.875237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:05.875237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:05.875237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:05.875237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:05.875237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:05.875237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:05.875237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:05.875237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:05.875237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:05.875237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:05.876261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:05.877218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:05.877218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:05.877218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:05.877218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:05.877218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:05.877218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:05.877218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:05.877218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:05.877218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:39:05.877218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:05.880246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 485/978] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:05.890921 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800da8c-86c3-4219-9aac-82fd7179b710 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:05.909171 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:06.009249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800da8c-86c3-4219-9aac-82fd7179b710 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:06.054010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafface7-dacd-44f1-a7d3-07f2c7d70651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:06.054556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafface7-dacd-44f1-a7d3-07f2c7d70651 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:06.054839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafface7-dacd-44f1-a7d3-07f2c7d70651 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:06.055176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafface7-dacd-44f1-a7d3-07f2c7d70651 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:06.055911 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafface7-dacd-44f1-a7d3-07f2c7d70651 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:06.056214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafface7-dacd-44f1-a7d3-07f2c7d70651 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:06.057680 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafface7-dacd-44f1-a7d3-07f2c7d70651 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:06.095917 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 494/979] 10.222.0.91 () {56 vars in 1101 bytes} [Thu Sep 8 13:39:05 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:06.550284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302251f9-aadf-4d07-8c50-4c43a7da5fba None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:06.551176 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302251f9-aadf-4d07-8c50-4c43a7da5fba None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:06.551393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302251f9-aadf-4d07-8c50-4c43a7da5fba None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:06.555732 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 486/980] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:06.561145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bde1b0-b416-4de6-ad3e-064f3cce75d4 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:06.561438 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bde1b0-b416-4de6-ad3e-064f3cce75d4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:06.561717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bde1b0-b416-4de6-ad3e-064f3cce75d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:06.580331 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-46bde1b0-b416-4de6-ad3e-064f3cce75d4 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:06.581526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-46bde1b0-b416-4de6-ad3e-064f3cce75d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:06.654058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 495/981] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:06.754970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447772e8-36fe-4fa8-b2ad-7b15a77d5db7 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:06.755276 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447772e8-36fe-4fa8-b2ad-7b15a77d5db7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:06.755449 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447772e8-36fe-4fa8-b2ad-7b15a77d5db7 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:06.756921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 487/982] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:06.762123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f3bb1-71e0-43e8-b6f4-27160ae58c36 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:06.762413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f3bb1-71e0-43e8-b6f4-27160ae58c36 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:06.762690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f3bb1-71e0-43e8-b6f4-27160ae58c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:06.770948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:06.783174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-223f3bb1-71e0-43e8-b6f4-27160ae58c36 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:06.784364 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-223f3bb1-71e0-43e8-b6f4-27160ae58c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:06.857785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 496/983] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:06.927864 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447772e8-36fe-4fa8-b2ad-7b15a77d5db7 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:07.033561 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447772e8-36fe-4fa8-b2ad-7b15a77d5db7 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:07.069010 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0db76-7ddf-4265-b027-5cb32296af5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:07.069613 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0db76-7ddf-4265-b027-5cb32296af5d None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.069893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0db76-7ddf-4265-b027-5cb32296af5d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.070165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0db76-7ddf-4265-b027-5cb32296af5d None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.070876 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0db76-7ddf-4265-b027-5cb32296af5d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:07.072340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0db76-7ddf-4265-b027-5cb32296af5d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:07.129018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f3bb1-71e0-43e8-b6f4-27160ae58c36 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:07.150298 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 488/984] 10.222.0.91 () {60 vars in 1141 bytes} [Thu Sep 8 13:39:06 2022] POST /identity/v3/services => generated 242 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:07.244642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f3bb1-71e0-43e8-b6f4-27160ae58c36 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:07.283915 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:07.291205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311768c-bc59-4ff8-a8d1-7aaab1ce4da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:07.291854 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311768c-bc59-4ff8-a8d1-7aaab1ce4da7 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.292128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311768c-bc59-4ff8-a8d1-7aaab1ce4da7 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.292644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311768c-bc59-4ff8-a8d1-7aaab1ce4da7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.310319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311768c-bc59-4ff8-a8d1-7aaab1ce4da7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:07.310319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311768c-bc59-4ff8-a8d1-7aaab1ce4da7 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:07.310319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-b311768c-bc59-4ff8-a8d1-7aaab1ce4da7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:07.310319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:07.310319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:07.310319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:07.310319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:07.310319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:07.310319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:07.310319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:07.310319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:07.310319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:07.310319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:07.311582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:07.312491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:07.313402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:07.313402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:07.313402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 497/985] 10.222.0.91 () {56 vars in 1126 bytes} [Thu Sep 8 13:39:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:07.313587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0db76-7ddf-4265-b027-5cb32296af5d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:07.319153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:07.427972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0db76-7ddf-4265-b027-5cb32296af5d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:07.462544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c7f80-bc28-4866-b5c1-231c1296beba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:07.463170 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c7f80-bc28-4866-b5c1-231c1296beba None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.463445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c7f80-bc28-4866-b5c1-231c1296beba None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.463711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c7f80-bc28-4866-b5c1-231c1296beba None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.464629 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c7f80-bc28-4866-b5c1-231c1296beba None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:07.465145 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c7f80-bc28-4866-b5c1-231c1296beba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:07.466577 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c7f80-bc28-4866-b5c1-231c1296beba None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:07.471714 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 489/986] 10.222.0.91 () {56 vars in 1122 bytes} [Thu Sep 8 13:39:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:07.484266 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311768c-bc59-4ff8-a8d1-7aaab1ce4da7 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:07.589159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311768c-bc59-4ff8-a8d1-7aaab1ce4da7 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:07.623125 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27af59-fb4c-481f-97d2-c05abfe2f868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:07.623730 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27af59-fb4c-481f-97d2-c05abfe2f868 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.624010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27af59-fb4c-481f-97d2-c05abfe2f868 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.624284 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27af59-fb4c-481f-97d2-c05abfe2f868 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.624910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27af59-fb4c-481f-97d2-c05abfe2f868 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:07.625214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27af59-fb4c-481f-97d2-c05abfe2f868 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:07.626585 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27af59-fb4c-481f-97d2-c05abfe2f868 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:07.636977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 498/987] 10.222.0.91 () {56 vars in 1122 bytes} [Thu Sep 8 13:39:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:07.638186 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c7f80-bc28-4866-b5c1-231c1296beba None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:07.709194 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:07.745553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c7f80-bc28-4866-b5c1-231c1296beba None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:07.781847 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286bf66-eff3-4a01-9509-df75ba09d4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:07.782442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286bf66-eff3-4a01-9509-df75ba09d4c7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.782716 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286bf66-eff3-4a01-9509-df75ba09d4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.783010 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286bf66-eff3-4a01-9509-df75ba09d4c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.783703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286bf66-eff3-4a01-9509-df75ba09d4c7 None admin] RBAC: Adding query filter params (interface=admin, service_id=5b87c5314d264435b9dfea5b1777d1b2, region_id=RegionOne) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:07.784005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286bf66-eff3-4a01-9509-df75ba09d4c7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:07.785908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286bf66-eff3-4a01-9509-df75ba09d4c7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:07.791166 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 490/988] 10.222.0.91 () {56 vars in 1249 bytes} [Thu Sep 8 13:39:07 2022] GET /identity/v3/endpoints?service_id=5b87c5314d264435b9dfea5b1777d1b2&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:07.796953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27af59-fb4c-481f-97d2-c05abfe2f868 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:07.863922 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:07.907970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27af59-fb4c-481f-97d2-c05abfe2f868 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:07.943678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6939b-ea8b-4608-a5fb-c919dca5015b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:07.944285 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6939b-ea8b-4608-a5fb-c919dca5015b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.944567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6939b-ea8b-4608-a5fb-c919dca5015b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.944843 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6939b-ea8b-4608-a5fb-c919dca5015b None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.945448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6939b-ea8b-4608-a5fb-c919dca5015b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:07.946908 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6939b-ea8b-4608-a5fb-c919dca5015b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:07.953944 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 499/989] 10.222.0.91 () {56 vars in 1087 bytes} [Thu Sep 8 13:39:07 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:39:08.158389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cc178-8dc7-412d-a796-bd38f7dcae5a None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.158877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cc178-8dc7-412d-a796-bd38f7dcae5a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.158877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cc178-8dc7-412d-a796-bd38f7dcae5a None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.160814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 491/990] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:08.166072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30623ecc-8539-4352-9ff3-8b88f089e49f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.166363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30623ecc-8539-4352-9ff3-8b88f089e49f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.166639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30623ecc-8539-4352-9ff3-8b88f089e49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.185076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-30623ecc-8539-4352-9ff3-8b88f089e49f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:08.186270 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-30623ecc-8539-4352-9ff3-8b88f089e49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:08.273287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 500/991] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:08.544264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cc178-8dc7-412d-a796-bd38f7dcae5a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:08.651923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cc178-8dc7-412d-a796-bd38f7dcae5a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:08.681514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:08.686864 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351fb3e-026b-4c7b-a89e-6a92fa474afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:08.687516 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351fb3e-026b-4c7b-a89e-6a92fa474afa None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.687791 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351fb3e-026b-4c7b-a89e-6a92fa474afa None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.688075 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351fb3e-026b-4c7b-a89e-6a92fa474afa None admin] PATH_INFO: `/v3/services/image` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.694863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351fb3e-026b-4c7b-a89e-6a92fa474afa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:08.694863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351fb3e-026b-4c7b-a89e-6a92fa474afa None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-a351fb3e-026b-4c7b-a89e-6a92fa474afa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:08.699211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:08.700178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:08.700178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:08.700178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:08.700178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:08.700178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:08.700178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:08.700178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:08.700178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:08.700178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:08.700178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:08.700178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:08.700178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:08.700178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:08.700178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:08.700178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:08.700178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:08.701022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:08.701022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:08.701022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:08.701022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:08.701022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:08.701022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:08.701022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:08.701022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:08.701022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:08.701022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:08.701022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:08.701022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:08.701022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:39:08.701022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:08.701022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 492/992] 10.222.0.91 () {56 vars in 1105 bytes} [Thu Sep 8 13:39:08 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:08.708266 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30623ecc-8539-4352-9ff3-8b88f089e49f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:08.766048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:08.815087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30623ecc-8539-4352-9ff3-8b88f089e49f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:08.850933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2aea55-0173-4ed6-ab39-7b92c8644dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:08.851551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2aea55-0173-4ed6-ab39-7b92c8644dac None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.851829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2aea55-0173-4ed6-ab39-7b92c8644dac None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.852099 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2aea55-0173-4ed6-ab39-7b92c8644dac None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.852730 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2aea55-0173-4ed6-ab39-7b92c8644dac None admin] RBAC: Adding query filter params (name=image) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:08.853029 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2aea55-0173-4ed6-ab39-7b92c8644dac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:08.854476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2aea55-0173-4ed6-ab39-7b92c8644dac None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:08.865486 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 501/993] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:08.866741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351fb3e-026b-4c7b-a89e-6a92fa474afa None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:08.975710 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351fb3e-026b-4c7b-a89e-6a92fa474afa None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:09.010937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1d1a2d-206f-4526-9893-527fb8936733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:09.011690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d1a2d-206f-4526-9893-527fb8936733 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.012016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d1a2d-206f-4526-9893-527fb8936733 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.012291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1d1a2d-206f-4526-9893-527fb8936733 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.012909 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d1a2d-206f-4526-9893-527fb8936733 None admin] RBAC: Adding query filter params (type=image) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:09.013206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d1a2d-206f-4526-9893-527fb8936733 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:09.014565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1d1a2d-206f-4526-9893-527fb8936733 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:09.027009 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 493/994] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:08 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:09.450705 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d953b-3a94-4541-bd79-6fd8f08d0c19 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.451084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d953b-3a94-4541-bd79-6fd8f08d0c19 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.451313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d953b-3a94-4541-bd79-6fd8f08d0c19 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.452876 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 502/995] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:09.459001 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3df3de-208c-4db0-aec3-bd9aaf2951d7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.459470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3df3de-208c-4db0-aec3-bd9aaf2951d7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.459547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3df3de-208c-4db0-aec3-bd9aaf2951d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.479788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-fb3df3de-208c-4db0-aec3-bd9aaf2951d7 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:09.481804 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fb3df3de-208c-4db0-aec3-bd9aaf2951d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:09.543654 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:09.555457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 494/996] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:09.824621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d953b-3a94-4541-bd79-6fd8f08d0c19 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:09.933512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d953b-3a94-4541-bd79-6fd8f08d0c19 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:09.951044 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:09.969595 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a4d00-bd4f-4d51-b898-179b4bfc30f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:09.970229 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a4d00-bd4f-4d51-b898-179b4bfc30f7 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.970508 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a4d00-bd4f-4d51-b898-179b4bfc30f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.970781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a4d00-bd4f-4d51-b898-179b4bfc30f7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.971392 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a4d00-bd4f-4d51-b898-179b4bfc30f7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:09.972794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a4d00-bd4f-4d51-b898-179b4bfc30f7 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-0a9a4d00-bd4f-4d51-b898-179b4bfc30f7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:09.981774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:09.982830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:09.982830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:09.982830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:09.982830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:09.982830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:09.982830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:09.982830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:09.982830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:09.982830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:09.982830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:09.982830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:09.982830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:09.982830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:09.982830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:09.982830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:09.982830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:09.983989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:09.983989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:09.983989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:09.983989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:09.983989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:09.983989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:09.983989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:09.983989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:09.983989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:09.983989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:09.983989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:09.983989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:09.983989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:09.983989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:09.987213 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 503/997] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:09.988741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3df3de-208c-4db0-aec3-bd9aaf2951d7 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.097396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3df3de-208c-4db0-aec3-bd9aaf2951d7 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.138601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba57a98-47d7-41d9-9110-aced57bc2459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.139280 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba57a98-47d7-41d9-9110-aced57bc2459 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.139563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba57a98-47d7-41d9-9110-aced57bc2459 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.139834 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba57a98-47d7-41d9-9110-aced57bc2459 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.140472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba57a98-47d7-41d9-9110-aced57bc2459 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:10.140776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba57a98-47d7-41d9-9110-aced57bc2459 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.142781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba57a98-47d7-41d9-9110-aced57bc2459 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.148392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 495/998] 10.222.0.91 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:10.153909 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a4d00-bd4f-4d51-b898-179b4bfc30f7 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.163549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df0fdc-1e47-45e8-804a-6349f11c338f None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.163830 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df0fdc-1e47-45e8-804a-6349f11c338f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.164098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df0fdc-1e47-45e8-804a-6349f11c338f None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.165569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 496/999] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:10.170344 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96ad11-6187-4d75-956c-b757b18b8f5d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.170622 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96ad11-6187-4d75-956c-b757b18b8f5d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.170941 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96ad11-6187-4d75-956c-b757b18b8f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.189357 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ad96ad11-6187-4d75-956c-b757b18b8f5d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:10.190540 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ad96ad11-6187-4d75-956c-b757b18b8f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:10.248200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:10.267664 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a4d00-bd4f-4d51-b898-179b4bfc30f7 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.268108 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 497/1000] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:10.303326 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd376973-0195-4706-9f85-c2c688a262cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.303923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd376973-0195-4706-9f85-c2c688a262cf None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.304202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd376973-0195-4706-9f85-c2c688a262cf None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.304466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd376973-0195-4706-9f85-c2c688a262cf None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.305083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd376973-0195-4706-9f85-c2c688a262cf None admin] RBAC: Adding query filter params (type=compute) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:10.305378 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd376973-0195-4706-9f85-c2c688a262cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.306729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd376973-0195-4706-9f85-c2c688a262cf None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.312343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 504/1001] 10.222.0.91 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:10.318584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96ad11-6187-4d75-956c-b757b18b8f5d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.419461 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96ad11-6187-4d75-956c-b757b18b8f5d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.455500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394e9b2-8e0d-4f0c-940c-b5f9b61350d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.456102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394e9b2-8e0d-4f0c-940c-b5f9b61350d9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.456374 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394e9b2-8e0d-4f0c-940c-b5f9b61350d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.456640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394e9b2-8e0d-4f0c-940c-b5f9b61350d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.457347 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394e9b2-8e0d-4f0c-940c-b5f9b61350d9 None admin] RBAC: Adding query filter params (interface=public, service_id=b6b4e6e8234e45fe9611b85947433a4d, region_id=RegionOne) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:10.457652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394e9b2-8e0d-4f0c-940c-b5f9b61350d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.459313 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394e9b2-8e0d-4f0c-940c-b5f9b61350d9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.464325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 498/1002] 10.222.0.91 () {56 vars in 1251 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/endpoints?service_id=b6b4e6e8234e45fe9611b85947433a4d&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:10.469878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd376973-0195-4706-9f85-c2c688a262cf None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.520002 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:10.534883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394e9b2-8e0d-4f0c-940c-b5f9b61350d9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.577264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd376973-0195-4706-9f85-c2c688a262cf None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.597111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:10.614065 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0502262-623e-4dc4-be51-3d0491140084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.614668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0502262-623e-4dc4-be51-3d0491140084 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.614971 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0502262-623e-4dc4-be51-3d0491140084 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.615242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0502262-623e-4dc4-be51-3d0491140084 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.616002 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0502262-623e-4dc4-be51-3d0491140084 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.617987 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0502262-623e-4dc4-be51-3d0491140084 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.625994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 505/1003] 10.222.0.91 () {56 vars in 1087 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:39:10.655761 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394e9b2-8e0d-4f0c-940c-b5f9b61350d9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.694584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57811717-25c5-4663-bd80-d095893563a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.695162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57811717-25c5-4663-bd80-d095893563a8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.695450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57811717-25c5-4663-bd80-d095893563a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.695702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57811717-25c5-4663-bd80-d095893563a8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.696291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57811717-25c5-4663-bd80-d095893563a8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.697674 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57811717-25c5-4663-bd80-d095893563a8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-57811717-25c5-4663-bd80-d095893563a8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:10.707818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:10.709111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:10.709111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:10.709111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:10.709111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:10.709111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:10.709111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:10.709111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:10.709111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:10.709111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:10.709111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:10.709111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:10.709111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:10.709111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:10.709111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:10.709111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:10.709111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:10.711913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:10.711913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:10.711913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:10.711913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:10.711913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:10.711913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:10.711913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:10.711913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:10.711913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:10.711913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:10.711913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:10.711913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:10.711913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:10.711913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:10.711913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 499/1004] 10.222.0.91 () {56 vars in 1126 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:10.715786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0502262-623e-4dc4-be51-3d0491140084 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.834164 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0502262-623e-4dc4-be51-3d0491140084 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.843943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:10.875463 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3804d35-1a95-49fa-a86d-78bc98e2952d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.876086 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3804d35-1a95-49fa-a86d-78bc98e2952d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.876369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3804d35-1a95-49fa-a86d-78bc98e2952d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.876643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3804d35-1a95-49fa-a86d-78bc98e2952d None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.877270 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3804d35-1a95-49fa-a86d-78bc98e2952d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:10.877661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3804d35-1a95-49fa-a86d-78bc98e2952d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.879708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3804d35-1a95-49fa-a86d-78bc98e2952d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.890752 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 506/1005] 10.222.0.91 () {56 vars in 1122 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:10.892727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57811717-25c5-4663-bd80-d095893563a8 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:11.000296 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57811717-25c5-4663-bd80-d095893563a8 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:11.013958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:11.035446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d73791-fe0d-4fd9-8451-9e77cb57d3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:11.036048 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d73791-fe0d-4fd9-8451-9e77cb57d3c2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.036326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d73791-fe0d-4fd9-8451-9e77cb57d3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.036597 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d73791-fe0d-4fd9-8451-9e77cb57d3c2 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.037221 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d73791-fe0d-4fd9-8451-9e77cb57d3c2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:11.037524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d73791-fe0d-4fd9-8451-9e77cb57d3c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:11.038906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d73791-fe0d-4fd9-8451-9e77cb57d3c2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:11.043854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 500/1006] 10.222.0.91 () {56 vars in 1122 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:11.049386 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3804d35-1a95-49fa-a86d-78bc98e2952d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:11.164682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3804d35-1a95-49fa-a86d-78bc98e2952d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:11.201490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5369ac-4622-4619-b1c9-51a20c2858b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:11.202158 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5369ac-4622-4619-b1c9-51a20c2858b3 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.202434 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5369ac-4622-4619-b1c9-51a20c2858b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.202700 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5369ac-4622-4619-b1c9-51a20c2858b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.203481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5369ac-4622-4619-b1c9-51a20c2858b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:11.204855 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5369ac-4622-4619-b1c9-51a20c2858b3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:11.249388 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ed09d-c9a6-4e12-906a-0db1f7a1c7c8 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.249672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ed09d-c9a6-4e12-906a-0db1f7a1c7c8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.249943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ed09d-c9a6-4e12-906a-0db1f7a1c7c8 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.251537 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 501/1007] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:11.256375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290ab89-6a35-4eb3-9140-acca9e36fc35 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.256659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290ab89-6a35-4eb3-9140-acca9e36fc35 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.256933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290ab89-6a35-4eb3-9140-acca9e36fc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.277430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-6290ab89-6a35-4eb3-9140-acca9e36fc35 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:11.279234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6290ab89-6a35-4eb3-9140-acca9e36fc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:11.322033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 507/1008] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:39:11 2022] POST /identity/v3/endpoints => generated 325 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:11.367550 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 502/1009] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:11 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:11.651683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5369ac-4622-4619-b1c9-51a20c2858b3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:11.686627 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:11.761601 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5369ac-4622-4619-b1c9-51a20c2858b3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:11.801448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad924b-a6fc-4ac9-9145-774720d17ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:11.802054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad924b-a6fc-4ac9-9145-774720d17ec1 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.802329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad924b-a6fc-4ac9-9145-774720d17ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.802599 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad924b-a6fc-4ac9-9145-774720d17ec1 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.803311 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad924b-a6fc-4ac9-9145-774720d17ec1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:11.804791 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad924b-a6fc-4ac9-9145-774720d17ec1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:11.947012 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 508/1010] 10.222.0.91 () {60 vars in 1141 bytes} [Thu Sep 8 13:39:11 2022] POST /identity/v3/services => generated 242 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:12.857998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a45ea-8632-48f0-95f6-b17789bb3a69 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:12.858506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a45ea-8632-48f0-95f6-b17789bb3a69 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:12.858584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a45ea-8632-48f0-95f6-b17789bb3a69 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:12.859975 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 503/1011] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:12.866288 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4235d6-de16-4584-bbc0-539267c1317f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:12.866642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4235d6-de16-4584-bbc0-539267c1317f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:12.867000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4235d6-de16-4584-bbc0-539267c1317f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:12.885984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-4e4235d6-de16-4584-bbc0-539267c1317f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:12.887182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4e4235d6-de16-4584-bbc0-539267c1317f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:12.971178 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 509/1012] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:12 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:13.241001 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a45ea-8632-48f0-95f6-b17789bb3a69 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:13.346133 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a45ea-8632-48f0-95f6-b17789bb3a69 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:13.347700 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:13.382559 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544e9a4-028c-4c82-9541-5d0208038d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:13.383780 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544e9a4-028c-4c82-9541-5d0208038d5c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.384437 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544e9a4-028c-4c82-9541-5d0208038d5c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.384964 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544e9a4-028c-4c82-9541-5d0208038d5c None admin] PATH_INFO: `/v3/services/compute` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.386022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544e9a4-028c-4c82-9541-5d0208038d5c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:13.387529 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544e9a4-028c-4c82-9541-5d0208038d5c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-0544e9a4-028c-4c82-9541-5d0208038d5c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:13.397509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:13.398586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:13.398586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:13.398586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:13.398586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:13.398586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:13.398586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:13.398586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:13.398586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:13.398586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:13.398586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:13.398586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:13.398586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:13.398586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:13.398586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:13.398586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:13.398586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:13.399867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:13.399867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:13.399867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:13.399867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:13.399867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:13.399867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:13.399867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:13.399867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:13.399867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:13.399867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:13.399867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:13.399867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:13.399867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:13.399867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:13.403315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 504/1013] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:13.405064 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4235d6-de16-4584-bbc0-539267c1317f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:13.426846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:13.516365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c178b-d19d-4b95-b649-02894f735037 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.516656 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c178b-d19d-4b95-b649-02894f735037 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.516842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c178b-d19d-4b95-b649-02894f735037 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.517905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4235d6-de16-4584-bbc0-539267c1317f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:13.518683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 505/1014] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:13.524966 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d6a6d-fc35-45e9-ba62-2553dbd0fc86 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.525257 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d6a6d-fc35-45e9-ba62-2553dbd0fc86 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.525535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d6a6d-fc35-45e9-ba62-2553dbd0fc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.547206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9e8d6a6d-fc35-45e9-ba62-2553dbd0fc86 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:13.548402 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9e8d6a6d-fc35-45e9-ba62-2553dbd0fc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:13.564829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c399f1-3d17-4392-b638-e58d961e9572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:13.565440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c399f1-3d17-4392-b638-e58d961e9572 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.565888 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c399f1-3d17-4392-b638-e58d961e9572 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.566167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c399f1-3d17-4392-b638-e58d961e9572 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.567202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c399f1-3d17-4392-b638-e58d961e9572 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:13.567537 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c399f1-3d17-4392-b638-e58d961e9572 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:13.568898 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c399f1-3d17-4392-b638-e58d961e9572 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:13.574981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 510/1015] 10.222.0.91 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:13.580786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c399f1-3d17-4392-b638-e58d961e9572 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:13.625830 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 506/1016] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:13 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:13.671164 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:13.689707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c399f1-3d17-4392-b638-e58d961e9572 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:13.726445 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487014f-b05c-4426-beef-e3c7dccd0052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:13.727079 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487014f-b05c-4426-beef-e3c7dccd0052 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.727361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487014f-b05c-4426-beef-e3c7dccd0052 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.727634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487014f-b05c-4426-beef-e3c7dccd0052 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.728256 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487014f-b05c-4426-beef-e3c7dccd0052 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:13.728559 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487014f-b05c-4426-beef-e3c7dccd0052 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:13.729923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487014f-b05c-4426-beef-e3c7dccd0052 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:13.735034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 511/1017] 10.222.0.91 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:13.740735 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d6a6d-fc35-45e9-ba62-2553dbd0fc86 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:13.840541 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:13.846784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d6a6d-fc35-45e9-ba62-2553dbd0fc86 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:13.882248 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa07df3-1f80-4b74-8811-37d07e23a565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:13.883076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa07df3-1f80-4b74-8811-37d07e23a565 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.883172 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa07df3-1f80-4b74-8811-37d07e23a565 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.883403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa07df3-1f80-4b74-8811-37d07e23a565 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.884070 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa07df3-1f80-4b74-8811-37d07e23a565 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:13.885744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa07df3-1f80-4b74-8811-37d07e23a565 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:13.897723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487014f-b05c-4426-beef-e3c7dccd0052 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:13.984469 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 507/1018] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:39:13 2022] POST /identity/v3/endpoints => generated 334 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:14.016509 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487014f-b05c-4426-beef-e3c7dccd0052 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.065374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522fa86-aee1-4d18-ba84-44fc9cc359be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.066016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522fa86-aee1-4d18-ba84-44fc9cc359be None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.066290 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522fa86-aee1-4d18-ba84-44fc9cc359be None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.067272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522fa86-aee1-4d18-ba84-44fc9cc359be None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.068419 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522fa86-aee1-4d18-ba84-44fc9cc359be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.069892 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522fa86-aee1-4d18-ba84-44fc9cc359be None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:14.074976 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 512/1019] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:14.077996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa07df3-1f80-4b74-8811-37d07e23a565 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.125850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a42d19-25b6-4ea6-9c20-7939fc17a4ed None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.126182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a42d19-25b6-4ea6-9c20-7939fc17a4ed None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.126394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a42d19-25b6-4ea6-9c20-7939fc17a4ed None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.127981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 513/1020] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:14.134159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ec3ba-4cef-41ff-967c-b38e2b3b8a9f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.134451 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ec3ba-4cef-41ff-967c-b38e2b3b8a9f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.134720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ec3ba-4cef-41ff-967c-b38e2b3b8a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.153780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-397ec3ba-4cef-41ff-967c-b38e2b3b8a9f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:14.155035 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-397ec3ba-4cef-41ff-967c-b38e2b3b8a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:14.159149 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:14.210552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa07df3-1f80-4b74-8811-37d07e23a565 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.244061 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 514/1021] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:14 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:14.252834 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ecaf2-2af5-42e1-ba8c-d491977f54b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.253708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ecaf2-2af5-42e1-ba8c-d491977f54b7 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.254569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ecaf2-2af5-42e1-ba8c-d491977f54b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.254844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ecaf2-2af5-42e1-ba8c-d491977f54b7 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.255563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ecaf2-2af5-42e1-ba8c-d491977f54b7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.257193 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ecaf2-2af5-42e1-ba8c-d491977f54b7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:14.363435 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 508/1022] 10.222.0.91 () {60 vars in 1141 bytes} [Thu Sep 8 13:39:14 2022] POST /identity/v3/projects => generated 311 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:14.514035 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ec3ba-4cef-41ff-967c-b38e2b3b8a9f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.627215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ec3ba-4cef-41ff-967c-b38e2b3b8a9f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.667627 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b28422-f6b7-45a8-a445-62682ef4f79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.668545 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b28422-f6b7-45a8-a445-62682ef4f79d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.668977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b28422-f6b7-45a8-a445-62682ef4f79d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.669384 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b28422-f6b7-45a8-a445-62682ef4f79d None admin] PATH_INFO: `/v3/services/image` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.670397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b28422-f6b7-45a8-a445-62682ef4f79d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.672397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b28422-f6b7-45a8-a445-62682ef4f79d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-09b28422-f6b7-45a8-a445-62682ef4f79d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:14.683948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:14.685132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:14.685132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:14.685132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:14.685132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:14.685132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:14.685132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:14.685132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:14.685132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:14.685132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:14.685132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:14.685132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:14.685132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:14.685132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:14.685132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:14.685132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:14.685132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:14.686141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:14.686141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:14.686141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:14.686141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:14.686141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:14.686141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:14.686141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:14.686141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:14.686141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:14.686141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:14.686141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:14.686141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:14.686141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:39:14.686141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:14.686141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 515/1023] 10.222.0.91 () {56 vars in 1105 bytes} [Thu Sep 8 13:39:14 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:14.691106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ecaf2-2af5-42e1-ba8c-d491977f54b7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.696478 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:14.796854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ecaf2-2af5-42e1-ba8c-d491977f54b7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.833021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38f380-9548-4f88-9c5a-d110133cd8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.833623 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38f380-9548-4f88-9c5a-d110133cd8d3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.834152 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38f380-9548-4f88-9c5a-d110133cd8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.834423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38f380-9548-4f88-9c5a-d110133cd8d3 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.835082 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38f380-9548-4f88-9c5a-d110133cd8d3 None admin] RBAC: Adding query filter params (name=image) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:14.835398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38f380-9548-4f88-9c5a-d110133cd8d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.836748 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38f380-9548-4f88-9c5a-d110133cd8d3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:14.842341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 509/1024] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:14.847927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b28422-f6b7-45a8-a445-62682ef4f79d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.851785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:14.960907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b28422-f6b7-45a8-a445-62682ef4f79d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.996976 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9434ecf-3684-4934-b421-a0ab2d67c91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.997582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9434ecf-3684-4934-b421-a0ab2d67c91b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.997851 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9434ecf-3684-4934-b421-a0ab2d67c91b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.998124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9434ecf-3684-4934-b421-a0ab2d67c91b None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.998761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9434ecf-3684-4934-b421-a0ab2d67c91b None admin] RBAC: Adding query filter params (type=image) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:14.999089 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9434ecf-3684-4934-b421-a0ab2d67c91b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:15.000471 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9434ecf-3684-4934-b421-a0ab2d67c91b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:15.012041 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 516/1025] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:15.013355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38f380-9548-4f88-9c5a-d110133cd8d3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:15.092686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:15.131216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38f380-9548-4f88-9c5a-d110133cd8d3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:15.166921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96265b9d-f9a9-4b14-aaa1-5a7d5a5e2ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:15.167527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96265b9d-f9a9-4b14-aaa1-5a7d5a5e2ccc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.167958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96265b9d-f9a9-4b14-aaa1-5a7d5a5e2ccc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.168619 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96265b9d-f9a9-4b14-aaa1-5a7d5a5e2ccc None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.168908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96265b9d-f9a9-4b14-aaa1-5a7d5a5e2ccc None admin] RBAC: Adding query filter params (interface=public, service_id=c76cbf5c39494166a66db8c2ec5bf496, region_id=RegionOne) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:15.169113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96265b9d-f9a9-4b14-aaa1-5a7d5a5e2ccc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:15.170447 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96265b9d-f9a9-4b14-aaa1-5a7d5a5e2ccc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:15.176298 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 510/1026] 10.222.0.91 () {56 vars in 1251 bytes} [Thu Sep 8 13:39:15 2022] GET /identity/v3/endpoints?service_id=c76cbf5c39494166a66db8c2ec5bf496&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:15.181939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9434ecf-3684-4934-b421-a0ab2d67c91b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:15.255156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:15.291516 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9434ecf-3684-4934-b421-a0ab2d67c91b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:15.327571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f79bab-acb1-409e-b09e-664a6c930bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:15.328182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f79bab-acb1-409e-b09e-664a6c930bbc None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.328521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f79bab-acb1-409e-b09e-664a6c930bbc None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.329047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f79bab-acb1-409e-b09e-664a6c930bbc None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.329596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f79bab-acb1-409e-b09e-664a6c930bbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:15.330774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f79bab-acb1-409e-b09e-664a6c930bbc None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:15.338292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 517/1027] 10.222.0.91 () {56 vars in 1087 bytes} [Thu Sep 8 13:39:15 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:39:16.321476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b545359-f0b2-45d9-b3b4-73519b209a4c None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:16.322002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b545359-f0b2-45d9-b3b4-73519b209a4c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:16.322002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b545359-f0b2-45d9-b3b4-73519b209a4c None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:16.323407 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 511/1028] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:16.328578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e295072-5e7a-45b8-99dd-c220b90ea97d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:16.328983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e295072-5e7a-45b8-99dd-c220b90ea97d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:16.329151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e295072-5e7a-45b8-99dd-c220b90ea97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:16.347149 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-8e295072-5e7a-45b8-99dd-c220b90ea97d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:16.348347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8e295072-5e7a-45b8-99dd-c220b90ea97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:16.428989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 518/1029] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:16.644101 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c065a75-faac-46fe-af8e-5291c18ee3ce None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:16.644442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c065a75-faac-46fe-af8e-5291c18ee3ce None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:16.644691 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c065a75-faac-46fe-af8e-5291c18ee3ce None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:16.646186 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 512/1030] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:16.651246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09801e56-36e8-425e-8cfa-e58c943ba732 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:16.651533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09801e56-36e8-425e-8cfa-e58c943ba732 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:16.651805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09801e56-36e8-425e-8cfa-e58c943ba732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:16.670446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-09801e56-36e8-425e-8cfa-e58c943ba732 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:16.671723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-09801e56-36e8-425e-8cfa-e58c943ba732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:16.697620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c065a75-faac-46fe-af8e-5291c18ee3ce None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:16.732561 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:16.744723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 519/1031] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:16.803749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c065a75-faac-46fe-af8e-5291c18ee3ce None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:16.834501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:16.839821 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b95ade-8d49-4409-973d-990c82df4735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:16.840440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b95ade-8d49-4409-973d-990c82df4735 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:16.840711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b95ade-8d49-4409-973d-990c82df4735 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:16.840974 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b95ade-8d49-4409-973d-990c82df4735 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:16.851670 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b95ade-8d49-4409-973d-990c82df4735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:16.856134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-d6b95ade-8d49-4409-973d-990c82df4735 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:16.856134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:16.856134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:16.856134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:16.856134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:16.856134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:16.856134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:16.856134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:16.856134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:16.856134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:16.856134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:16.856134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:16.856134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:16.856134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:16.856134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:16.856134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:16.857218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:16.857218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:16.857218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:16.857218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:16.857218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:16.857218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:16.857218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:16.857218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:16.857218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:16.857218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:16.857218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:16.857218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:16.857218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:16.857218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:16.857218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:16.857218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:16.858094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:16.858094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:16.858094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:16.858094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:16.858094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:16.858094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:16.858094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:16.858094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:16.858094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:16.858094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:16.858094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:16.858094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:16.862245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 513/1032] 10.222.0.91 () {56 vars in 1120 bytes} [Thu Sep 8 13:39:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:16.864079 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09801e56-36e8-425e-8cfa-e58c943ba732 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:16.968310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09801e56-36e8-425e-8cfa-e58c943ba732 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:17.002718 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346975d-4659-42bc-bdf8-adca3670de4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:17.003355 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346975d-4659-42bc-bdf8-adca3670de4e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.003621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346975d-4659-42bc-bdf8-adca3670de4e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.003890 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346975d-4659-42bc-bdf8-adca3670de4e None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.005097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346975d-4659-42bc-bdf8-adca3670de4e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:17.005391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346975d-4659-42bc-bdf8-adca3670de4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:17.006742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346975d-4659-42bc-bdf8-adca3670de4e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:17.020372 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 520/1033] 10.222.0.91 () {56 vars in 1115 bytes} [Thu Sep 8 13:39:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:17.020574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b95ade-8d49-4409-973d-990c82df4735 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:17.022051 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346975d-4659-42bc-bdf8-adca3670de4e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:17.081799 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:17.122538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b95ade-8d49-4409-973d-990c82df4735 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:17.134018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346975d-4659-42bc-bdf8-adca3670de4e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:17.167652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72cc3e-47d1-47cf-9be2-2866fb1e4fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:17.168385 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72cc3e-47d1-47cf-9be2-2866fb1e4fa7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.168660 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72cc3e-47d1-47cf-9be2-2866fb1e4fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.168924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72cc3e-47d1-47cf-9be2-2866fb1e4fa7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.170010 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72cc3e-47d1-47cf-9be2-2866fb1e4fa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:17.171824 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72cc3e-47d1-47cf-9be2-2866fb1e4fa7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:17.174027 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 514/1034] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:17.176216 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c7ced-544c-4d2f-b914-bddb37409b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:17.176849 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c7ced-544c-4d2f-b914-bddb37409b4c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.177125 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c7ced-544c-4d2f-b914-bddb37409b4c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.177397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c7ced-544c-4d2f-b914-bddb37409b4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.179640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72cc3e-47d1-47cf-9be2-2866fb1e4fa7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:17.188197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c7ced-544c-4d2f-b914-bddb37409b4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:17.189665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c7ced-544c-4d2f-b914-bddb37409b4c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:17.205657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-c30c7ced-544c-4d2f-b914-bddb37409b4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:17.205657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:17.205657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:17.205657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:17.205657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:17.205657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:17.205657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:17.205657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:17.205657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:17.205657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:17.205657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:17.205657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:17.205657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:17.205657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:17.205657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:17.205657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:17.206684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:17.208687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:17.208687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:17.208687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:17.208687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 521/1035] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:17.213605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c7ced-544c-4d2f-b914-bddb37409b4c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:17.249912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:17.298426 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72cc3e-47d1-47cf-9be2-2866fb1e4fa7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:17.322884 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c7ced-544c-4d2f-b914-bddb37409b4c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:17.329990 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:17.339371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1194f1-daf4-45bb-8dd7-512e6ed68fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:17.340063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1194f1-daf4-45bb-8dd7-512e6ed68fac None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.340346 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1194f1-daf4-45bb-8dd7-512e6ed68fac None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.340621 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1194f1-daf4-45bb-8dd7-512e6ed68fac None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.341357 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1194f1-daf4-45bb-8dd7-512e6ed68fac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:17.342822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1194f1-daf4-45bb-8dd7-512e6ed68fac None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:17.359871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e4bcc-8bdc-4c40-b42e-a1fd1ef84e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:17.360706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e4bcc-8bdc-4c40-b42e-a1fd1ef84e2d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.361921 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e4bcc-8bdc-4c40-b42e-a1fd1ef84e2d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.361921 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e4bcc-8bdc-4c40-b42e-a1fd1ef84e2d None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.362315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e4bcc-8bdc-4c40-b42e-a1fd1ef84e2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:17.362608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e4bcc-8bdc-4c40-b42e-a1fd1ef84e2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:17.364089 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e4bcc-8bdc-4c40-b42e-a1fd1ef84e2d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:17.375918 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 522/1036] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:17.382020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e4bcc-8bdc-4c40-b42e-a1fd1ef84e2d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:17.447177 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 515/1037] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:39:17 2022] POST /identity/v3/users => generated 285 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:39:17.508288 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e4bcc-8bdc-4c40-b42e-a1fd1ef84e2d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:17.530061 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70282348-c62a-4c47-8ca4-15a13e02c83f None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.530418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70282348-c62a-4c47-8ca4-15a13e02c83f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.530661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70282348-c62a-4c47-8ca4-15a13e02c83f None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.532077 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 516/1038] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:17.538311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f2e83-5517-4396-931b-c552fdf13053 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.538655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f2e83-5517-4396-931b-c552fdf13053 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.538861 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f2e83-5517-4396-931b-c552fdf13053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.548803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedb3b1-9f5b-4d06-b058-4e35dba3fadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:17.549435 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedb3b1-9f5b-4d06-b058-4e35dba3fadb None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.549719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedb3b1-9f5b-4d06-b058-4e35dba3fadb None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.549993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedb3b1-9f5b-4d06-b058-4e35dba3fadb None admin] PATH_INFO: `/v3/users/nova` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.559661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-213f2e83-5517-4396-931b-c552fdf13053 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:17.559909 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bedb3b1-9f5b-4d06-b058-4e35dba3fadb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:17.561195 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bedb3b1-9f5b-4d06-b058-4e35dba3fadb None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:17.561619 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-213f2e83-5517-4396-931b-c552fdf13053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:17.573506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-3bedb3b1-9f5b-4d06-b058-4e35dba3fadb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:17.573506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:17.573506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:17.573506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:17.573506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:17.573506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:17.573506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:17.573506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:17.573506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:17.573506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:17.573506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:17.573506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:17.573506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:17.573506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:17.573506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:17.573506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:17.574679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:17.575766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:17.575766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:17.575766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:17.575766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:17.575766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:17.575766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:17.575766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:17.575766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:17.575766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:17.575766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:17.575766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 523/1039] 10.222.0.91 () {56 vars in 1093 bytes} [Thu Sep 8 13:39:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:17.580683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedb3b1-9f5b-4d06-b058-4e35dba3fadb None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:17.613348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:17.645433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 517/1040] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:17.695623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedb3b1-9f5b-4d06-b058-4e35dba3fadb None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:17.734206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9829e7-1fac-41d9-b92b-3cfc76bf9205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:17.734810 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9829e7-1fac-41d9-b92b-3cfc76bf9205 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.735119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9829e7-1fac-41d9-b92b-3cfc76bf9205 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.735391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9829e7-1fac-41d9-b92b-3cfc76bf9205 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.736117 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9829e7-1fac-41d9-b92b-3cfc76bf9205 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:17.736417 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9829e7-1fac-41d9-b92b-3cfc76bf9205 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:17.737850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9829e7-1fac-41d9-b92b-3cfc76bf9205 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:17.758821 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 524/1041] 10.222.0.91 () {56 vars in 1133 bytes} [Thu Sep 8 13:39:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:17.764543 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f2e83-5517-4396-931b-c552fdf13053 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:17.812524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:17.868066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f2e83-5517-4396-931b-c552fdf13053 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:17.903810 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf7cb1-4ae8-4793-8fb5-537ced5da050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:17.904414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf7cb1-4ae8-4793-8fb5-537ced5da050 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.904688 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf7cb1-4ae8-4793-8fb5-537ced5da050 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.904958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf7cb1-4ae8-4793-8fb5-537ced5da050 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.920761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9829e7-1fac-41d9-b92b-3cfc76bf9205 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:17.982468 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf7cb1-4ae8-4793-8fb5-537ced5da050 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:17.983954 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf7cb1-4ae8-4793-8fb5-537ced5da050 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.014769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.029966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9829e7-1fac-41d9-b92b-3cfc76bf9205 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.034465 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.069308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f2f2e-5206-4c33-a264-d1b49ae71b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.070622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f2f2e-5206-4c33-a264-d1b49ae71b26 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.071418 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f2f2e-5206-4c33-a264-d1b49ae71b26 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.071891 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f2f2e-5206-4c33-a264-d1b49ae71b26 None admin] PATH_INFO: `/v3/services/image` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.072903 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f2f2e-5206-4c33-a264-d1b49ae71b26 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.075049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f2f2e-5206-4c33-a264-d1b49ae71b26 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-123f2f2e-5206-4c33-a264-d1b49ae71b26 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:18.091245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:18.092517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:18.092517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:18.092517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:18.092517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:18.092517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:18.092517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:18.092517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:18.092517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:18.092517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:18.092517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:18.092517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:18.092517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:18.092517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:18.092517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:18.092517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:18.092517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:18.094637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:18.094637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:18.094637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:18.094637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:18.094637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:18.094637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:18.094637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:18.094637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:18.094637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:18.094637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:18.094637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:18.094637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:18.094637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:39:18.094637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:18.094637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 525/1042] 10.222.0.91 () {56 vars in 1105 bytes} [Thu Sep 8 13:39:17 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:18.099380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 518/1043] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:17 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:39:18.099942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f2f2e-5206-4c33-a264-d1b49ae71b26 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.104695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf7cb1-4ae8-4793-8fb5-537ced5da050 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.211733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf7cb1-4ae8-4793-8fb5-537ced5da050 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.216325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f2f2e-5206-4c33-a264-d1b49ae71b26 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.254211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2136b6b-0aa2-4b2a-9a2e-3c69dca7683c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.254455 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb297448-ddb5-4fc2-a1e6-0e76b4743dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.254841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2136b6b-0aa2-4b2a-9a2e-3c69dca7683c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.255068 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb297448-ddb5-4fc2-a1e6-0e76b4743dff None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.255184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2136b6b-0aa2-4b2a-9a2e-3c69dca7683c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.255383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb297448-ddb5-4fc2-a1e6-0e76b4743dff None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.255484 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2136b6b-0aa2-4b2a-9a2e-3c69dca7683c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.255707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb297448-ddb5-4fc2-a1e6-0e76b4743dff None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.256337 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb297448-ddb5-4fc2-a1e6-0e76b4743dff None admin] RBAC: Adding query filter params (name=image) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:18.256640 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb297448-ddb5-4fc2-a1e6-0e76b4743dff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.258016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb297448-ddb5-4fc2-a1e6-0e76b4743dff None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.267005 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 526/1044] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:18.268836 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb297448-ddb5-4fc2-a1e6-0e76b4743dff None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.270608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2136b6b-0aa2-4b2a-9a2e-3c69dca7683c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.272101 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2136b6b-0aa2-4b2a-9a2e-3c69dca7683c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.279932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.284051 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.288739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-c2136b6b-0aa2-4b2a-9a2e-3c69dca7683c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:18.288739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:18.288739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:18.288739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:18.288739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:18.288739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:18.288739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:18.288739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:18.288739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:18.288739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:18.288739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:18.288739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:18.288739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:18.288739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:18.288739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:18.288739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:18.289696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:18.302048 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:18.302048 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:18.302048 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:18.302048 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 519/1045] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:18.302048 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2136b6b-0aa2-4b2a-9a2e-3c69dca7683c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.378874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb297448-ddb5-4fc2-a1e6-0e76b4743dff None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.408310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2136b6b-0aa2-4b2a-9a2e-3c69dca7683c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.423222 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd9143-bc9f-4aa1-987e-f625c4866f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.423832 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd9143-bc9f-4aa1-987e-f625c4866f50 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.424108 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd9143-bc9f-4aa1-987e-f625c4866f50 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.424378 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd9143-bc9f-4aa1-987e-f625c4866f50 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.425009 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd9143-bc9f-4aa1-987e-f625c4866f50 None admin] RBAC: Adding query filter params (type=image) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:18.425311 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd9143-bc9f-4aa1-987e-f625c4866f50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.426706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd9143-bc9f-4aa1-987e-f625c4866f50 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.436053 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 527/1046] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:18.437347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd9143-bc9f-4aa1-987e-f625c4866f50 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.450394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21e07b-ed02-4f97-9922-982638b555bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.451368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21e07b-ed02-4f97-9922-982638b555bd None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.451799 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21e07b-ed02-4f97-9922-982638b555bd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.452340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21e07b-ed02-4f97-9922-982638b555bd None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.453275 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21e07b-ed02-4f97-9922-982638b555bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:18.453583 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21e07b-ed02-4f97-9922-982638b555bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.455024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21e07b-ed02-4f97-9922-982638b555bd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.471452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 520/1047] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:18.479930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21e07b-ed02-4f97-9922-982638b555bd None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.525731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.543013 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.549626 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd9143-bc9f-4aa1-987e-f625c4866f50 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.585537 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7999e-e608-40a1-b44e-87dbf5b74ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.586139 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7999e-e608-40a1-b44e-87dbf5b74ee1 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.586417 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7999e-e608-40a1-b44e-87dbf5b74ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.586690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7999e-e608-40a1-b44e-87dbf5b74ee1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.587378 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7999e-e608-40a1-b44e-87dbf5b74ee1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.587966 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21e07b-ed02-4f97-9922-982638b555bd None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.588752 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7999e-e608-40a1-b44e-87dbf5b74ee1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.623724 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acdbcb-ec31-4064-8d92-f33895c8abe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.624772 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acdbcb-ec31-4064-8d92-f33895c8abe6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.624772 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acdbcb-ec31-4064-8d92-f33895c8abe6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.624950 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acdbcb-ec31-4064-8d92-f33895c8abe6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.632159 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acdbcb-ec31-4064-8d92-f33895c8abe6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.633630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acdbcb-ec31-4064-8d92-f33895c8abe6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.644630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-98acdbcb-ec31-4064-8d92-f33895c8abe6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:18.644630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:18.644630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:18.644630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:18.644630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:18.644630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:18.644630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:18.644630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:18.644630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:18.644630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:18.644630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:18.644630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:18.644630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:18.644630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:18.644630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:18.647701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:18.648646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:18.648646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:18.648646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:18.648646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:18.648646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:18.648646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:18.648646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:18.648646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:18.648646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 521/1048] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:18.651530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acdbcb-ec31-4064-8d92-f33895c8abe6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.670478 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 528/1049] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:39:18 2022] POST /identity/v3/endpoints => generated 327 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:39:18.776372 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acdbcb-ec31-4064-8d92-f33895c8abe6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.796253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.823132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee4a2d-42f2-457a-ac9b-1a420031e0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.823747 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee4a2d-42f2-457a-ac9b-1a420031e0c4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.824032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee4a2d-42f2-457a-ac9b-1a420031e0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.824304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee4a2d-42f2-457a-ac9b-1a420031e0c4 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.825163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee4a2d-42f2-457a-ac9b-1a420031e0c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:18.825355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee4a2d-42f2-457a-ac9b-1a420031e0c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.827966 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee4a2d-42f2-457a-ac9b-1a420031e0c4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.846948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 522/1050] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:18.848771 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7999e-e608-40a1-b44e-87dbf5b74ee1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.965931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7999e-e608-40a1-b44e-87dbf5b74ee1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.995105 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:19.000677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a609311-b184-4c92-8201-182a77ba0fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:19.009668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a609311-b184-4c92-8201-182a77ba0fa7 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:19.009668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a609311-b184-4c92-8201-182a77ba0fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:19.009668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a609311-b184-4c92-8201-182a77ba0fa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:19.009668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a609311-b184-4c92-8201-182a77ba0fa7 None admin] RBAC: Adding query filter params (role.id=cf5de1e5771c4b60911d58b68de13600, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=38127f41852d412ea6ad270f1618fce2) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:19.009668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a609311-b184-4c92-8201-182a77ba0fa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:19.009668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a609311-b184-4c92-8201-182a77ba0fa7 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:19.010866 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 529/1051] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/role_assignments?user.id=38127f41852d412ea6ad270f1618fce2&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=cf5de1e5771c4b60911d58b68de13600 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:19.783143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36afcdf-9eba-4402-ab30-19f0cd3c15ae None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:19.783669 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36afcdf-9eba-4402-ab30-19f0cd3c15ae None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:19.783669 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36afcdf-9eba-4402-ab30-19f0cd3c15ae None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:19.785080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 523/1052] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:19.790965 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dc684-eaea-4e4d-9c28-873a19d5a2ba None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:19.791381 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dc684-eaea-4e4d-9c28-873a19d5a2ba None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:19.791612 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dc684-eaea-4e4d-9c28-873a19d5a2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:19.811261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-021dc684-eaea-4e4d-9c28-873a19d5a2ba None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:19.812470 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-021dc684-eaea-4e4d-9c28-873a19d5a2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:19.893879 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 530/1053] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:20.163346 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36afcdf-9eba-4402-ab30-19f0cd3c15ae None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:20.272843 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36afcdf-9eba-4402-ab30-19f0cd3c15ae None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:20.310863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b3178-93b1-432a-94c2-ab52193244f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:20.311913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b3178-93b1-432a-94c2-ab52193244f9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.312444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b3178-93b1-432a-94c2-ab52193244f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.312933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b3178-93b1-432a-94c2-ab52193244f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.324063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b3178-93b1-432a-94c2-ab52193244f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:20.329165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-720b3178-93b1-432a-94c2-ab52193244f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:20.329165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:20.329165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:20.329165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:20.329165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:20.329165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:20.329165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:20.329165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:20.329165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:20.329165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:20.329165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:20.329165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:20.329165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:20.329165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:20.329165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:20.329165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:20.330090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:20.330090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:20.330090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:20.330090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:20.330090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:20.330090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:20.330090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:20.330090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:20.330090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:20.330090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:20.330090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:20.330090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:20.330090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:20.330090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:20.330090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:20.330090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:20.330980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:20.330980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:20.330980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:20.330980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:20.330980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:20.330980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:20.330980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:20.330980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:20.330980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:20.330980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:20.330980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:20.330980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:20.334095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 524/1054] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:20.340446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dc684-eaea-4e4d-9c28-873a19d5a2ba None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:20.446993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dc684-eaea-4e4d-9c28-873a19d5a2ba None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:20.483105 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:20.492670 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e6c1a-d59d-4a60-ab7d-c7ed4315509c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:20.496137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e6c1a-d59d-4a60-ab7d-c7ed4315509c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.496137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e6c1a-d59d-4a60-ab7d-c7ed4315509c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.496137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e6c1a-d59d-4a60-ab7d-c7ed4315509c None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.496137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e6c1a-d59d-4a60-ab7d-c7ed4315509c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:20.496137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e6c1a-d59d-4a60-ab7d-c7ed4315509c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:20.496137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e6c1a-d59d-4a60-ab7d-c7ed4315509c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:20.505252 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 531/1055] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:20.511207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b3178-93b1-432a-94c2-ab52193244f9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:20.547085 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:20.615523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b3178-93b1-432a-94c2-ab52193244f9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:20.650758 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209928e7-27ab-46f1-9ec9-13d85d7a5d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:20.651414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209928e7-27ab-46f1-9ec9-13d85d7a5d66 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.651694 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209928e7-27ab-46f1-9ec9-13d85d7a5d66 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.651965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209928e7-27ab-46f1-9ec9-13d85d7a5d66 None admin] PATH_INFO: `/v3/users/4ab95a4201b447fd8f42098f1a010900` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.666457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209928e7-27ab-46f1-9ec9-13d85d7a5d66 None admin] RBAC: Authorizing `identity:get_user(user_id=4ab95a4201b447fd8f42098f1a010900)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:20.667915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209928e7-27ab-46f1-9ec9-13d85d7a5d66 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:20.669736 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 525/1056] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:20 2022] GET /identity/v3/users/4ab95a4201b447fd8f42098f1a010900 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:20.675546 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e6c1a-d59d-4a60-ab7d-c7ed4315509c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:20.782616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e6c1a-d59d-4a60-ab7d-c7ed4315509c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:20.826347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ad1a7-4a85-44ba-893d-76640f530e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:20.827037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ad1a7-4a85-44ba-893d-76640f530e87 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.827280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ad1a7-4a85-44ba-893d-76640f530e87 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.827550 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ad1a7-4a85-44ba-893d-76640f530e87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.889665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:20.905871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ad1a7-4a85-44ba-893d-76640f530e87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:20.908141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ad1a7-4a85-44ba-893d-76640f530e87 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:20.928329 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff279b28-88e2-4cdb-a731-98ff0daf9680 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.928883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff279b28-88e2-4cdb-a731-98ff0daf9680 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.928883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff279b28-88e2-4cdb-a731-98ff0daf9680 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.930388 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 526/1057] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:20.935841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d001be8-8d6b-4fd7-aa4a-f45dd4776da6 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.936139 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d001be8-8d6b-4fd7-aa4a-f45dd4776da6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.936531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d001be8-8d6b-4fd7-aa4a-f45dd4776da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.955731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9d001be8-8d6b-4fd7-aa4a-f45dd4776da6 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:20.957031 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9d001be8-8d6b-4fd7-aa4a-f45dd4776da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:21.030226 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 527/1058] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:21.036351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 532/1059] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:39:21.037453 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d001be8-8d6b-4fd7-aa4a-f45dd4776da6 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.064113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:21.149197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d001be8-8d6b-4fd7-aa4a-f45dd4776da6 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.180279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba155a-2eb2-4f79-ad57-6758f15f0f30 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.180579 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba155a-2eb2-4f79-ad57-6758f15f0f30 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.180833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba155a-2eb2-4f79-ad57-6758f15f0f30 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.182432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 533/1060] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:21.186737 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec72f7-adf4-4f69-81c9-4e7bd5a3e9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.187332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e1a8a-b41c-4637-853d-a93874948020 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.187440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec72f7-adf4-4f69-81c9-4e7bd5a3e9e1 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.187618 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e1a8a-b41c-4637-853d-a93874948020 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.187722 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec72f7-adf4-4f69-81c9-4e7bd5a3e9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.187897 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e1a8a-b41c-4637-853d-a93874948020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.188002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec72f7-adf4-4f69-81c9-4e7bd5a3e9e1 None admin] PATH_INFO: `/v3/projects/0aed73c3c18141ec8d49a3aeaa87b7bd` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.189113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec72f7-adf4-4f69-81c9-4e7bd5a3e9e1 None admin] RBAC: Authorizing `identity:get_project(project_id=0aed73c3c18141ec8d49a3aeaa87b7bd)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:21.190568 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec72f7-adf4-4f69-81c9-4e7bd5a3e9e1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:21.192379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 528/1061] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:21 2022] GET /identity/v3/projects/0aed73c3c18141ec8d49a3aeaa87b7bd => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:21.198337 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec72f7-adf4-4f69-81c9-4e7bd5a3e9e1 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.207761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-860e1a8a-b41c-4637-853d-a93874948020 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:21.208988 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-860e1a8a-b41c-4637-853d-a93874948020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:21.285228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 534/1062] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:21.294962 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:21.304487 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec72f7-adf4-4f69-81c9-4e7bd5a3e9e1 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.310076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e1a8a-b41c-4637-853d-a93874948020 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.326394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:21.340067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a76f63-694c-411e-ab0b-7ce40a2630c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.340668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a76f63-694c-411e-ab0b-7ce40a2630c1 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.340942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a76f63-694c-411e-ab0b-7ce40a2630c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.343068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a76f63-694c-411e-ab0b-7ce40a2630c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.343068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a76f63-694c-411e-ab0b-7ce40a2630c1 None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=0aed73c3c18141ec8d49a3aeaa87b7bd, user.id=4ab95a4201b447fd8f42098f1a010900) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:21.343068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a76f63-694c-411e-ab0b-7ce40a2630c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:21.343708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a76f63-694c-411e-ab0b-7ce40a2630c1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:21.349875 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 529/1063] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:21 2022] GET /identity/v3/role_assignments?user.id=4ab95a4201b447fd8f42098f1a010900&scope.project.id=0aed73c3c18141ec8d49a3aeaa87b7bd&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:21.416068 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e1a8a-b41c-4637-853d-a93874948020 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.452532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c57c74-c8bf-4ad4-8cf1-1a8746c4ff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.453154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c57c74-c8bf-4ad4-8cf1-1a8746c4ff97 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.453427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c57c74-c8bf-4ad4-8cf1-1a8746c4ff97 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.453694 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c57c74-c8bf-4ad4-8cf1-1a8746c4ff97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.537200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c57c74-c8bf-4ad4-8cf1-1a8746c4ff97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:21.538661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c57c74-c8bf-4ad4-8cf1-1a8746c4ff97 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:21.559490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:21.573678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a76f63-694c-411e-ab0b-7ce40a2630c1 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.647721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 535/1064] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 13:39:21.654180 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c57c74-c8bf-4ad4-8cf1-1a8746c4ff97 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.688779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a76f63-694c-411e-ab0b-7ce40a2630c1 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.728737 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9c7c1-cad5-468c-8eb0-273d5fabee7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.729839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9c7c1-cad5-468c-8eb0-273d5fabee7d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.729839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9c7c1-cad5-468c-8eb0-273d5fabee7d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.730054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9c7c1-cad5-468c-8eb0-273d5fabee7d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.741889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9c7c1-cad5-468c-8eb0-273d5fabee7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:21.746860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-f3a9c7c1-cad5-468c-8eb0-273d5fabee7d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:21.746860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:21.746860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:21.746860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:21.746860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:21.746860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:21.746860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:21.746860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:21.746860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:21.746860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:21.746860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:21.746860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:21.746860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:21.746860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:21.746860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:21.746860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:21.747814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:21.747814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:21.747814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:21.747814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:21.747814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:21.747814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:21.747814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:21.747814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:21.747814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:21.747814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:21.747814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:21.747814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:21.747814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:21.747814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:21.747814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:21.747814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:21.748865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:21.748865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:21.748865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:21.748865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:21.748865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:21.748865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:21.748865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:21.748865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:21.748865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:21.748865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:21.748865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:21.748865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:21.748865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 530/1065] 10.222.0.91 () {56 vars in 1120 bytes} [Thu Sep 8 13:39:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:21.754121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9c7c1-cad5-468c-8eb0-273d5fabee7d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.763042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c57c74-c8bf-4ad4-8cf1-1a8746c4ff97 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.770966 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:21.780686 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:21.800248 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385387c-d716-44c7-b05f-d15deb921613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.800883 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385387c-d716-44c7-b05f-d15deb921613 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.801206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385387c-d716-44c7-b05f-d15deb921613 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.801960 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385387c-d716-44c7-b05f-d15deb921613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.812544 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385387c-d716-44c7-b05f-d15deb921613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:21.814002 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385387c-d716-44c7-b05f-d15deb921613 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:21.830107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-b385387c-d716-44c7-b05f-d15deb921613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:21.830107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:21.830107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:21.830107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:21.830107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:21.830107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:21.830107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:21.830107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:21.830107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:21.830107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:21.830107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:21.830107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:21.830107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:21.830107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:21.830107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:21.830107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:21.833241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:21.834616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:21.834616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:21.834616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:21.836523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 536/1066] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:21.837391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385387c-d716-44c7-b05f-d15deb921613 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.870710 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9c7c1-cad5-468c-8eb0-273d5fabee7d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.906713 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c42ad-c182-4b01-bfe1-79190f2a081b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.907435 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c42ad-c182-4b01-bfe1-79190f2a081b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.907713 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c42ad-c182-4b01-bfe1-79190f2a081b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.907977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c42ad-c182-4b01-bfe1-79190f2a081b None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.908743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c42ad-c182-4b01-bfe1-79190f2a081b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:21.909044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c42ad-c182-4b01-bfe1-79190f2a081b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:21.910393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c42ad-c182-4b01-bfe1-79190f2a081b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:21.923240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 531/1067] 10.222.0.91 () {56 vars in 1115 bytes} [Thu Sep 8 13:39:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:21.925148 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c42ad-c182-4b01-bfe1-79190f2a081b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.944208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385387c-d716-44c7-b05f-d15deb921613 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.978829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7055f-e58c-41cc-9b9c-14d5cae385af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.979479 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7055f-e58c-41cc-9b9c-14d5cae385af None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.979765 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7055f-e58c-41cc-9b9c-14d5cae385af None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.980038 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7055f-e58c-41cc-9b9c-14d5cae385af None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.980677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7055f-e58c-41cc-9b9c-14d5cae385af None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:21.980980 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7055f-e58c-41cc-9b9c-14d5cae385af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:21.982354 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7055f-e58c-41cc-9b9c-14d5cae385af None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:21.994788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 537/1068] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:22.022630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:22.048330 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c42ad-c182-4b01-bfe1-79190f2a081b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:22.089339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4a778-14d3-49d3-9d2a-7a0ac18cd311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:22.089906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4a778-14d3-49d3-9d2a-7a0ac18cd311 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:22.090268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4a778-14d3-49d3-9d2a-7a0ac18cd311 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:22.090543 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4a778-14d3-49d3-9d2a-7a0ac18cd311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:22.105446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4a778-14d3-49d3-9d2a-7a0ac18cd311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:22.107005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4a778-14d3-49d3-9d2a-7a0ac18cd311 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:22.123457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-2cf4a778-14d3-49d3-9d2a-7a0ac18cd311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:22.123457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:22.123457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:22.123457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:22.123457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:22.123457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:22.123457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:22.123457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:22.123457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:22.123457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:22.123457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:22.123457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:22.123457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:22.123457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:22.123457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:22.123457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:22.124493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:22.131687 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:22.131687 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:22.131687 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:22.131687 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 532/1069] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:22.132134 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7055f-e58c-41cc-9b9c-14d5cae385af None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:22.171915 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:22.255175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7055f-e58c-41cc-9b9c-14d5cae385af None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:22.293200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47b108-b81f-4488-8b3e-9958c0af2360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:22.293821 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47b108-b81f-4488-8b3e-9958c0af2360 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:22.294108 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47b108-b81f-4488-8b3e-9958c0af2360 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:22.294389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47b108-b81f-4488-8b3e-9958c0af2360 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:22.295058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47b108-b81f-4488-8b3e-9958c0af2360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:22.295368 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47b108-b81f-4488-8b3e-9958c0af2360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:22.296811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47b108-b81f-4488-8b3e-9958c0af2360 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:22.312935 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 538/1070] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:22.319547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4a778-14d3-49d3-9d2a-7a0ac18cd311 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:22.419848 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4a778-14d3-49d3-9d2a-7a0ac18cd311 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:22.445695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:22.453012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164a31c-f860-4103-a3df-4699210ffc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:22.453627 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164a31c-f860-4103-a3df-4699210ffc85 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:22.453904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164a31c-f860-4103-a3df-4699210ffc85 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:22.454175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164a31c-f860-4103-a3df-4699210ffc85 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:22.462347 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164a31c-f860-4103-a3df-4699210ffc85 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:22.463806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164a31c-f860-4103-a3df-4699210ffc85 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:22.475339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-a164a31c-f860-4103-a3df-4699210ffc85 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:22.475339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:22.475339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:22.475339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:22.475339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:22.475339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:22.475339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:22.475339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:22.475339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:22.475339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:22.475339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:22.475339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:22.475339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:22.475339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:22.475339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:22.475339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:22.476305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:22.477290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:22.477290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:22.477290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:22.477290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:22.477290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:22.477290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:22.477290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:22.477290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:22.477290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:22.477290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:22.477290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 533/1071] 10.222.0.91 () {56 vars in 1093 bytes} [Thu Sep 8 13:39:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:22.482493 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47b108-b81f-4488-8b3e-9958c0af2360 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:22.573317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:22.582743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47b108-b81f-4488-8b3e-9958c0af2360 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:22.617226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398ba06-1ee0-4887-8dac-117fcebc80b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:22.617825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398ba06-1ee0-4887-8dac-117fcebc80b5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:22.618103 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398ba06-1ee0-4887-8dac-117fcebc80b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:22.618371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398ba06-1ee0-4887-8dac-117fcebc80b5 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:22.619107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398ba06-1ee0-4887-8dac-117fcebc80b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:22.619407 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398ba06-1ee0-4887-8dac-117fcebc80b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:22.620833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398ba06-1ee0-4887-8dac-117fcebc80b5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:22.637581 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 539/1072] 10.222.0.91 () {56 vars in 1133 bytes} [Thu Sep 8 13:39:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:22.643382 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164a31c-f860-4103-a3df-4699210ffc85 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:22.741497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164a31c-f860-4103-a3df-4699210ffc85 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:22.773070 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb4c15-4bce-48cb-93d8-ed6d87ab81c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:22.773777 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb4c15-4bce-48cb-93d8-ed6d87ab81c8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:22.774054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb4c15-4bce-48cb-93d8-ed6d87ab81c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:22.774317 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb4c15-4bce-48cb-93d8-ed6d87ab81c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:22.784394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb4c15-4bce-48cb-93d8-ed6d87ab81c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:22.786334 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb4c15-4bce-48cb-93d8-ed6d87ab81c8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:22.799755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-3beb4c15-4bce-48cb-93d8-ed6d87ab81c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:22.799755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:22.799755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:22.799755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:22.799755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:22.799755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:22.799755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:22.799755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:22.799755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:22.799755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:22.799755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:22.799755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:22.799755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:22.799755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:22.799755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:22.799755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:22.800854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:22.802279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:22.802279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:22.802279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:22.802279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 534/1073] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:22.807402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398ba06-1ee0-4887-8dac-117fcebc80b5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:22.905235 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398ba06-1ee0-4887-8dac-117fcebc80b5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:22.939832 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafd8e2-c310-4287-9574-f30f0a9a1125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:22.940429 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafd8e2-c310-4287-9574-f30f0a9a1125 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:22.940704 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafd8e2-c310-4287-9574-f30f0a9a1125 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:22.940971 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafd8e2-c310-4287-9574-f30f0a9a1125 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:22.941598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafd8e2-c310-4287-9574-f30f0a9a1125 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:22.941925 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafd8e2-c310-4287-9574-f30f0a9a1125 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:22.943315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafd8e2-c310-4287-9574-f30f0a9a1125 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:22.955485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 540/1074] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:22.961877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb4c15-4bce-48cb-93d8-ed6d87ab81c8 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:23.002990 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:23.059625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb4c15-4bce-48cb-93d8-ed6d87ab81c8 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:23.093266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417a7e3-ab39-49a0-832b-f445e86ab4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:23.093880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417a7e3-ab39-49a0-832b-f445e86ab4b4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:23.094151 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417a7e3-ab39-49a0-832b-f445e86ab4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:23.094418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417a7e3-ab39-49a0-832b-f445e86ab4b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:23.101414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417a7e3-ab39-49a0-832b-f445e86ab4b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:23.102836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417a7e3-ab39-49a0-832b-f445e86ab4b4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:23.114207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-c417a7e3-ab39-49a0-832b-f445e86ab4b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:23.114207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:23.114207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:23.114207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:23.114207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:23.114207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:23.114207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:23.114207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:23.114207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:23.114207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:23.114207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:23.114207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:23.114207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:23.114207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:23.114207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:23.115351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:23.116440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:23.116440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:23.116440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:23.116440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:23.116440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:23.116440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:23.116440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:23.116440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:23.119039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 535/1075] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:23.121136 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafd8e2-c310-4287-9574-f30f0a9a1125 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:23.132353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:23.225073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafd8e2-c310-4287-9574-f30f0a9a1125 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:23.261642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47104db-c503-4c95-93ab-62fbefde2781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:23.262224 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47104db-c503-4c95-93ab-62fbefde2781 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:23.262501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47104db-c503-4c95-93ab-62fbefde2781 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:23.262776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47104db-c503-4c95-93ab-62fbefde2781 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:23.263504 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47104db-c503-4c95-93ab-62fbefde2781 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:23.263811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47104db-c503-4c95-93ab-62fbefde2781 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:23.265266 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47104db-c503-4c95-93ab-62fbefde2781 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:23.278685 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 541/1076] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:23.284827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417a7e3-ab39-49a0-832b-f445e86ab4b4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:23.388342 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:23.391496 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417a7e3-ab39-49a0-832b-f445e86ab4b4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:23.436003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29007b9b-c9cc-421f-87f1-b6f90c83a123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:23.436684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29007b9b-c9cc-421f-87f1-b6f90c83a123 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:23.436963 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29007b9b-c9cc-421f-87f1-b6f90c83a123 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:23.437237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29007b9b-c9cc-421f-87f1-b6f90c83a123 None admin] PATH_INFO: `/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/38127f41852d412ea6ad270f1618fce2/roles/cf5de1e5771c4b60911d58b68de13600` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:23.439544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29007b9b-c9cc-421f-87f1-b6f90c83a123 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb2bd53d40841eeb1653bd5927fb189, user_id=38127f41852d412ea6ad270f1618fce2, role_id=cf5de1e5771c4b60911d58b68de13600)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:23.441021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29007b9b-c9cc-421f-87f1-b6f90c83a123 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:23.505354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 536/1077] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:39:23 2022] PUT /identity/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/38127f41852d412ea6ad270f1618fce2/roles/cf5de1e5771c4b60911d58b68de13600 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:39:23.557592 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb2a9e-ae22-4bff-86f4-1c6432ff04d1 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:23.557820 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb2a9e-ae22-4bff-86f4-1c6432ff04d1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:23.558026 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb2a9e-ae22-4bff-86f4-1c6432ff04d1 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:23.559559 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 542/1078] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:23.565544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39b98c-2d9d-488b-95e1-1a41f847059e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:23.565835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39b98c-2d9d-488b-95e1-1a41f847059e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:23.566114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39b98c-2d9d-488b-95e1-1a41f847059e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:23.585689 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0c39b98c-2d9d-488b-95e1-1a41f847059e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:23.586994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0c39b98c-2d9d-488b-95e1-1a41f847059e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:23.660598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 537/1079] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:23.929316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb2a9e-ae22-4bff-86f4-1c6432ff04d1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:23.999019 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:24.062406 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb2a9e-ae22-4bff-86f4-1c6432ff04d1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:24.071381 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad78c0-9edf-4255-b3ed-89bac9398647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:24.072006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad78c0-9edf-4255-b3ed-89bac9398647 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.072282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad78c0-9edf-4255-b3ed-89bac9398647 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.072549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad78c0-9edf-4255-b3ed-89bac9398647 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.082496 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad78c0-9edf-4255-b3ed-89bac9398647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:24.087497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-33ad78c0-9edf-4255-b3ed-89bac9398647 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:24.087497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:24.087497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:24.087497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:24.087497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:24.087497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:24.087497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:24.087497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:24.087497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:24.087497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:24.087497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:24.087497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:24.087497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:24.087497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:24.087497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:24.087497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:24.088636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:24.088636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:24.088636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:24.088636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:24.088636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:24.088636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:24.088636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:24.088636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:24.088636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:24.088636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:24.088636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:24.088636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:24.088636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:24.088636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:24.088636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:24.088636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:24.089869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:24.089869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:24.089869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:24.089869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:24.089869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:24.089869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:24.089869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:24.089869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:24.089869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:24.089869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:24.089869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:24.089869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:24.089869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 543/1080] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:24.094472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39b98c-2d9d-488b-95e1-1a41f847059e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:24.196835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:24.207057 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39b98c-2d9d-488b-95e1-1a41f847059e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:24.243798 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010c5eb-0959-44ac-9165-81643514921b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:24.244389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010c5eb-0959-44ac-9165-81643514921b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.244662 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010c5eb-0959-44ac-9165-81643514921b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.244928 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010c5eb-0959-44ac-9165-81643514921b None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.245574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010c5eb-0959-44ac-9165-81643514921b None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:24.245870 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010c5eb-0959-44ac-9165-81643514921b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:24.247260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010c5eb-0959-44ac-9165-81643514921b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:24.256012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 538/1081] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:24.261809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad78c0-9edf-4255-b3ed-89bac9398647 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:24.363050 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad78c0-9edf-4255-b3ed-89bac9398647 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:24.390733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:24.399148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203ebaa-20da-4a25-9256-36c04845d6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:24.399782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203ebaa-20da-4a25-9256-36c04845d6e0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.403544 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203ebaa-20da-4a25-9256-36c04845d6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.403544 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203ebaa-20da-4a25-9256-36c04845d6e0 None admin] PATH_INFO: `/v3/users/4ab95a4201b447fd8f42098f1a010900` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.403544 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203ebaa-20da-4a25-9256-36c04845d6e0 None admin] RBAC: Authorizing `identity:get_user(user_id=4ab95a4201b447fd8f42098f1a010900)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:24.403544 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203ebaa-20da-4a25-9256-36c04845d6e0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:24.404606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 544/1082] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:24 2022] GET /identity/v3/users/4ab95a4201b447fd8f42098f1a010900 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:24.410394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010c5eb-0959-44ac-9165-81643514921b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:24.510752 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010c5eb-0959-44ac-9165-81643514921b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:24.543973 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e33f3-1dca-40e7-887c-1e1b932483b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:24.544693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e33f3-1dca-40e7-887c-1e1b932483b1 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.544969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e33f3-1dca-40e7-887c-1e1b932483b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.545240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e33f3-1dca-40e7-887c-1e1b932483b1 None admin] PATH_INFO: `/v3/projects/0aed73c3c18141ec8d49a3aeaa87b7bd` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.546344 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e33f3-1dca-40e7-887c-1e1b932483b1 None admin] RBAC: Authorizing `identity:get_project(project_id=0aed73c3c18141ec8d49a3aeaa87b7bd)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:24.547797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e33f3-1dca-40e7-887c-1e1b932483b1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:24.549422 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 539/1083] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:24 2022] GET /identity/v3/projects/0aed73c3c18141ec8d49a3aeaa87b7bd => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:24.555860 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203ebaa-20da-4a25-9256-36c04845d6e0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:24.661872 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203ebaa-20da-4a25-9256-36c04845d6e0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:24.698661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed83cac-844a-4e2b-a96b-1c3e144bab5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:24.699352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed83cac-844a-4e2b-a96b-1c3e144bab5e None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.699626 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed83cac-844a-4e2b-a96b-1c3e144bab5e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.699895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed83cac-844a-4e2b-a96b-1c3e144bab5e None admin] PATH_INFO: `/v3/projects/0aed73c3c18141ec8d49a3aeaa87b7bd/users/4ab95a4201b447fd8f42098f1a010900/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.703014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed83cac-844a-4e2b-a96b-1c3e144bab5e None admin] RBAC: Authorizing `identity:create_grant(project_id=0aed73c3c18141ec8d49a3aeaa87b7bd, user_id=4ab95a4201b447fd8f42098f1a010900, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:24.704042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed83cac-844a-4e2b-a96b-1c3e144bab5e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:24.824513 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 545/1084] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:39:24 2022] PUT /identity/v3/projects/0aed73c3c18141ec8d49a3aeaa87b7bd/users/4ab95a4201b447fd8f42098f1a010900/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:39:25.665311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacdc97-8c31-40c9-987a-cb9b409ecac2 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.665833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacdc97-8c31-40c9-987a-cb9b409ecac2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.665833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacdc97-8c31-40c9-987a-cb9b409ecac2 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.667374 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 540/1085] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:25.672506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e75869-2008-4906-9a3f-4fa72250359d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.672925 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e75869-2008-4906-9a3f-4fa72250359d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.673122 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e75869-2008-4906-9a3f-4fa72250359d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.695384 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-79e75869-2008-4906-9a3f-4fa72250359d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:25.696609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-79e75869-2008-4906-9a3f-4fa72250359d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:25.754228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:25.771348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 546/1086] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:26.037892 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacdc97-8c31-40c9-987a-cb9b409ecac2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:26.061365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:26.145967 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacdc97-8c31-40c9-987a-cb9b409ecac2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:26.180285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5be12e-e286-4a94-9754-9ff544e1ac68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:26.180907 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5be12e-e286-4a94-9754-9ff544e1ac68 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:26.181184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5be12e-e286-4a94-9754-9ff544e1ac68 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:26.181455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5be12e-e286-4a94-9754-9ff544e1ac68 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:26.192969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5be12e-e286-4a94-9754-9ff544e1ac68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:26.197805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-ef5be12e-e286-4a94-9754-9ff544e1ac68 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:26.197805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:26.197805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:26.197805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:26.197805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:26.197805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:26.197805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:26.197805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:26.197805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:26.197805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:26.197805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:26.197805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:26.197805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:26.197805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:26.197805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:26.197805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:26.198794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:26.198794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:26.198794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:26.198794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:26.198794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:26.198794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:26.198794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:26.198794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:26.198794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:26.198794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:26.198794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:26.198794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:26.198794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:26.198794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:26.198794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:26.198794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:26.199746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:26.199746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:26.199746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:26.199746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:26.199746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:26.199746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:26.199746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:26.199746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:26.199746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:26.199746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:26.199746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:26.199746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:26.199746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 541/1087] 10.222.0.91 () {56 vars in 1120 bytes} [Thu Sep 8 13:39:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:26.205178 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e75869-2008-4906-9a3f-4fa72250359d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:26.305652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e75869-2008-4906-9a3f-4fa72250359d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:26.339121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254db94-1008-4fbd-a9fe-9eea3b6e353d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:26.339717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254db94-1008-4fbd-a9fe-9eea3b6e353d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:26.339993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254db94-1008-4fbd-a9fe-9eea3b6e353d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:26.340566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e254db94-1008-4fbd-a9fe-9eea3b6e353d None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:26.341339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254db94-1008-4fbd-a9fe-9eea3b6e353d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:26.341640 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254db94-1008-4fbd-a9fe-9eea3b6e353d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:26.343024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e254db94-1008-4fbd-a9fe-9eea3b6e353d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:26.351551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 547/1088] 10.222.0.91 () {56 vars in 1115 bytes} [Thu Sep 8 13:39:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:26.357380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5be12e-e286-4a94-9754-9ff544e1ac68 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:26.437649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:26.456323 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5be12e-e286-4a94-9754-9ff544e1ac68 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:26.493653 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f162c3-d836-42fe-836e-eea7a8c83a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:26.494275 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f162c3-d836-42fe-836e-eea7a8c83a8f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:26.494554 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f162c3-d836-42fe-836e-eea7a8c83a8f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:26.494823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f162c3-d836-42fe-836e-eea7a8c83a8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:26.505258 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f162c3-d836-42fe-836e-eea7a8c83a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:26.506688 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f162c3-d836-42fe-836e-eea7a8c83a8f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:26.520625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-91f162c3-d836-42fe-836e-eea7a8c83a8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:26.520625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:26.520625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:26.520625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:26.520625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:26.520625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:26.520625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:26.520625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:26.520625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:26.520625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:26.520625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:26.520625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:26.520625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:26.520625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:26.520625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:26.520625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:26.521894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:26.522877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:26.522877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:26.522877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:26.522877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 542/1089] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:26.527947 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254db94-1008-4fbd-a9fe-9eea3b6e353d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:26.579059 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:26.634594 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e254db94-1008-4fbd-a9fe-9eea3b6e353d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:26.668776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdced3-787b-4a9a-955f-68b7c3201c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:26.669381 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdced3-787b-4a9a-955f-68b7c3201c29 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:26.669658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdced3-787b-4a9a-955f-68b7c3201c29 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:26.669928 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdced3-787b-4a9a-955f-68b7c3201c29 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:26.670572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdced3-787b-4a9a-955f-68b7c3201c29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:26.670873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdced3-787b-4a9a-955f-68b7c3201c29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:26.672337 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdced3-787b-4a9a-955f-68b7c3201c29 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:26.684673 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 548/1090] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:26.690520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f162c3-d836-42fe-836e-eea7a8c83a8f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:26.793530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f162c3-d836-42fe-836e-eea7a8c83a8f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:26.821943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:26.827384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac479e-9760-41e2-8fc8-dc5bfbb1488f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:26.828019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac479e-9760-41e2-8fc8-dc5bfbb1488f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:26.828297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac479e-9760-41e2-8fc8-dc5bfbb1488f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:26.828570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac479e-9760-41e2-8fc8-dc5bfbb1488f None admin] PATH_INFO: `/v3/users/nova` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:26.837144 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac479e-9760-41e2-8fc8-dc5bfbb1488f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:26.838565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac479e-9760-41e2-8fc8-dc5bfbb1488f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:26.851125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-52ac479e-9760-41e2-8fc8-dc5bfbb1488f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:26.851125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:26.851125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:26.851125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:26.851125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:26.851125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:26.851125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:26.851125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:26.851125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:26.851125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:26.851125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:26.851125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:26.851125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:26.851125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:26.851125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:26.851125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:26.852106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:26.858026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:26.858026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:26.858026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:26.858026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:26.858026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:26.858026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:26.858026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:26.858026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:26.858026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:26.858026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:26.858026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 543/1091] 10.222.0.91 () {56 vars in 1093 bytes} [Thu Sep 8 13:39:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:26.858755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdced3-787b-4a9a-955f-68b7c3201c29 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:26.966523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdced3-787b-4a9a-955f-68b7c3201c29 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:26.977512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:27.001509 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2132b5-d0f2-4fc0-b031-bffafde0c790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:27.002179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2132b5-d0f2-4fc0-b031-bffafde0c790 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:27.002460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2132b5-d0f2-4fc0-b031-bffafde0c790 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:27.003042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2132b5-d0f2-4fc0-b031-bffafde0c790 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:27.003412 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0bc28-8215-457d-b03d-f1b268a2ee83 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:27.003692 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2132b5-d0f2-4fc0-b031-bffafde0c790 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:27.003787 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0bc28-8215-457d-b03d-f1b268a2ee83 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:27.003963 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0bc28-8215-457d-b03d-f1b268a2ee83 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:27.004065 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2132b5-d0f2-4fc0-b031-bffafde0c790 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:27.005813 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 544/1092] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:27.006070 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2132b5-d0f2-4fc0-b031-bffafde0c790 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:27.010793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47b7f6-8c6c-4780-a330-dd43cb7a9e3e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:27.011102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47b7f6-8c6c-4780-a330-dd43cb7a9e3e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:27.011436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47b7f6-8c6c-4780-a330-dd43cb7a9e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:27.026110 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 549/1093] 10.222.0.91 () {56 vars in 1133 bytes} [Thu Sep 8 13:39:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:27.031197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8e47b7f6-8c6c-4780-a330-dd43cb7a9e3e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:27.032072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2132b5-d0f2-4fc0-b031-bffafde0c790 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:27.032397 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8e47b7f6-8c6c-4780-a330-dd43cb7a9e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:27.104302 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 545/1094] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:27.148778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2132b5-d0f2-4fc0-b031-bffafde0c790 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:27.191373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16c98f-1b1c-4676-9282-89500d98e1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:27.191971 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16c98f-1b1c-4676-9282-89500d98e1a3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:27.192780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16c98f-1b1c-4676-9282-89500d98e1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:27.193043 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16c98f-1b1c-4676-9282-89500d98e1a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:27.229799 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:27.265850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16c98f-1b1c-4676-9282-89500d98e1a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:27.267360 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16c98f-1b1c-4676-9282-89500d98e1a3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:27.368259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 550/1095] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 13:39:27.373918 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47b7f6-8c6c-4780-a330-dd43cb7a9e3e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:27.378221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16c98f-1b1c-4676-9282-89500d98e1a3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:27.442710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:27.475266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47b7f6-8c6c-4780-a330-dd43cb7a9e3e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:27.480335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16c98f-1b1c-4676-9282-89500d98e1a3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:27.498835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:27.510546 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38910b6e-e591-411a-bfd0-2335ac2367e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:27.511199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38910b6e-e591-411a-bfd0-2335ac2367e3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:27.511476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38910b6e-e591-411a-bfd0-2335ac2367e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:27.511742 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38910b6e-e591-411a-bfd0-2335ac2367e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:27.515698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9426219-9030-493b-bfe3-651df7482c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:27.516767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9426219-9030-493b-bfe3-651df7482c38 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:27.517073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9426219-9030-493b-bfe3-651df7482c38 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:27.517356 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9426219-9030-493b-bfe3-651df7482c38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:27.521724 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38910b6e-e591-411a-bfd0-2335ac2367e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:27.523209 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38910b6e-e591-411a-bfd0-2335ac2367e3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:27.527360 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9426219-9030-493b-bfe3-651df7482c38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:27.532636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-f9426219-9030-493b-bfe3-651df7482c38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:27.532636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:27.532636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:27.532636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:27.532636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:27.532636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:27.532636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:27.532636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:27.532636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:27.532636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:27.532636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:27.532636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:27.532636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:27.532636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:27.532636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:27.532636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:27.533773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:27.533773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:27.533773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:27.533773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:27.533773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:27.533773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:27.533773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:27.533773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:27.533773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:27.533773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:27.533773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:27.533773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:27.533773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:27.533773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:27.533773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:27.533773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:27.534842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:27.534842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:27.534842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:27.534842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:27.534842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:27.534842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:27.534842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:27.534842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:27.534842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:27.534842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:27.534842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:27.534842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:27.534842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 551/1096] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:27.539962 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9426219-9030-493b-bfe3-651df7482c38 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:27.540436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-38910b6e-e591-411a-bfd0-2335ac2367e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:27.540436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:27.540436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:27.540436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:27.540436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:27.540436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:27.540436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:27.540436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:27.540436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:27.540436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:27.540436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:27.540436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:27.540436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:27.540436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:27.540436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:27.540436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:27.541563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:27.542518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:27.542518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:27.542518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:27.542518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 546/1097] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:27.547134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38910b6e-e591-411a-bfd0-2335ac2367e3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:27.650589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9426219-9030-493b-bfe3-651df7482c38 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:27.652814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38910b6e-e591-411a-bfd0-2335ac2367e3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:27.676826 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:27.685619 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33a0f0-dfb7-49d2-9540-88040bba9a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:27.686268 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33a0f0-dfb7-49d2-9540-88040bba9a14 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:27.686550 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33a0f0-dfb7-49d2-9540-88040bba9a14 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:27.689759 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33a0f0-dfb7-49d2-9540-88040bba9a14 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:27.689759 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33a0f0-dfb7-49d2-9540-88040bba9a14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:27.689930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a47223-55de-427e-a3a6-6bec1d3b4825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:27.690019 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33a0f0-dfb7-49d2-9540-88040bba9a14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:27.690098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a47223-55de-427e-a3a6-6bec1d3b4825 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:27.690098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a47223-55de-427e-a3a6-6bec1d3b4825 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:27.690098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a47223-55de-427e-a3a6-6bec1d3b4825 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:27.690273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c33a0f0-dfb7-49d2-9540-88040bba9a14 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:27.690341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a47223-55de-427e-a3a6-6bec1d3b4825 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:27.690341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a47223-55de-427e-a3a6-6bec1d3b4825 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:27.691045 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a47223-55de-427e-a3a6-6bec1d3b4825 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:27.698226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 552/1098] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:27.703358 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 547/1099] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:27.704749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33a0f0-dfb7-49d2-9540-88040bba9a14 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:27.709258 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a47223-55de-427e-a3a6-6bec1d3b4825 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:27.744583 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:27.810206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33a0f0-dfb7-49d2-9540-88040bba9a14 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:27.817589 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a47223-55de-427e-a3a6-6bec1d3b4825 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:27.849473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf3433-3e82-49de-8de7-56a5ad534a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:27.850112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf3433-3e82-49de-8de7-56a5ad534a38 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:27.850385 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf3433-3e82-49de-8de7-56a5ad534a38 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:27.850653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf3433-3e82-49de-8de7-56a5ad534a38 None admin] PATH_INFO: `/v3/users/4ab95a4201b447fd8f42098f1a010900` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:27.851913 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf3433-3e82-49de-8de7-56a5ad534a38 None admin] RBAC: Authorizing `identity:get_user(user_id=4ab95a4201b447fd8f42098f1a010900)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:27.853380 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf3433-3e82-49de-8de7-56a5ad534a38 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:27.855053 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 553/1100] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:27 2022] GET /identity/v3/users/4ab95a4201b447fd8f42098f1a010900 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:27.856007 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d52be-059a-42b3-b634-0ebbd1538e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:27.856785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d52be-059a-42b3-b634-0ebbd1538e16 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:27.857009 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d52be-059a-42b3-b634-0ebbd1538e16 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:27.857280 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d52be-059a-42b3-b634-0ebbd1538e16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:27.861486 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf3433-3e82-49de-8de7-56a5ad534a38 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:27.865792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d52be-059a-42b3-b634-0ebbd1538e16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:27.867288 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d52be-059a-42b3-b634-0ebbd1538e16 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:27.880113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-433d52be-059a-42b3-b634-0ebbd1538e16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:27.880113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:27.880113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:27.880113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:27.880113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:27.880113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:27.880113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:27.880113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:27.880113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:27.880113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:27.880113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:27.880113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:27.880113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:27.880113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:27.880113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:27.881430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:27.882345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:27.882345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:27.882345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:27.882345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:27.882345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:27.882345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:27.882345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:27.882345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:27.882345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 548/1101] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:27.887402 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d52be-059a-42b3-b634-0ebbd1538e16 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:27.922389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:27.966059 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf3433-3e82-49de-8de7-56a5ad534a38 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:27.984553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:27.991068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d52be-059a-42b3-b634-0ebbd1538e16 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:28.002130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bdf48-a22a-4bc1-a692-a841ee7f4981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:28.002735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bdf48-a22a-4bc1-a692-a841ee7f4981 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:28.003048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bdf48-a22a-4bc1-a692-a841ee7f4981 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:28.003323 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bdf48-a22a-4bc1-a692-a841ee7f4981 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:28.025333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064a5e5-0978-4e8a-8edf-b5c722cbba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:28.025925 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064a5e5-0978-4e8a-8edf-b5c722cbba91 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:28.026201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064a5e5-0978-4e8a-8edf-b5c722cbba91 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:28.026472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064a5e5-0978-4e8a-8edf-b5c722cbba91 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:28.027187 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064a5e5-0978-4e8a-8edf-b5c722cbba91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:28.027491 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064a5e5-0978-4e8a-8edf-b5c722cbba91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:28.028943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064a5e5-0978-4e8a-8edf-b5c722cbba91 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:28.040921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 549/1102] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:28.046920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064a5e5-0978-4e8a-8edf-b5c722cbba91 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:28.073005 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96bdf48-a22a-4bc1-a692-a841ee7f4981 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:28.074492 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96bdf48-a22a-4bc1-a692-a841ee7f4981 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:28.129639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:28.145718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064a5e5-0978-4e8a-8edf-b5c722cbba91 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:28.179007 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 554/1103] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:39:28.179557 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c1600-6b64-4975-8e33-712b993b9c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:28.180175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c1600-6b64-4975-8e33-712b993b9c85 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:28.180463 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c1600-6b64-4975-8e33-712b993b9c85 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:28.180615 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bdf48-a22a-4bc1-a692-a841ee7f4981 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:28.180733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c1600-6b64-4975-8e33-712b993b9c85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:28.181452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c1600-6b64-4975-8e33-712b993b9c85 None admin] RBAC: Adding query filter params (role.id=cf5de1e5771c4b60911d58b68de13600, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=38127f41852d412ea6ad270f1618fce2) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:28.181765 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c1600-6b64-4975-8e33-712b993b9c85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:28.183260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c1600-6b64-4975-8e33-712b993b9c85 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:28.190925 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 550/1104] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:28 2022] GET /identity/v3/role_assignments?user.id=38127f41852d412ea6ad270f1618fce2&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=cf5de1e5771c4b60911d58b68de13600 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:28.287310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bdf48-a22a-4bc1-a692-a841ee7f4981 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:28.323549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844509c-a2de-4b9c-aec5-fd2f192a3c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:28.324549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844509c-a2de-4b9c-aec5-fd2f192a3c12 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:28.324549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844509c-a2de-4b9c-aec5-fd2f192a3c12 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:28.324766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844509c-a2de-4b9c-aec5-fd2f192a3c12 None admin] PATH_INFO: `/v3/projects/0aed73c3c18141ec8d49a3aeaa87b7bd` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:28.325943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844509c-a2de-4b9c-aec5-fd2f192a3c12 None admin] RBAC: Authorizing `identity:get_project(project_id=0aed73c3c18141ec8d49a3aeaa87b7bd)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:28.327589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844509c-a2de-4b9c-aec5-fd2f192a3c12 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:28.329358 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 555/1105] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:28 2022] GET /identity/v3/projects/0aed73c3c18141ec8d49a3aeaa87b7bd => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:28.336631 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c1600-6b64-4975-8e33-712b993b9c85 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:28.358709 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:28.443970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c1600-6b64-4975-8e33-712b993b9c85 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:28.481629 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f8b18-62c2-4ff7-a197-aa2e07f87202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:28.482237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f8b18-62c2-4ff7-a197-aa2e07f87202 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:28.482512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f8b18-62c2-4ff7-a197-aa2e07f87202 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:28.482781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f8b18-62c2-4ff7-a197-aa2e07f87202 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:28.483957 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f8b18-62c2-4ff7-a197-aa2e07f87202 None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=0aed73c3c18141ec8d49a3aeaa87b7bd, user.id=4ab95a4201b447fd8f42098f1a010900) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:28.483957 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f8b18-62c2-4ff7-a197-aa2e07f87202 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:28.485328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f8b18-62c2-4ff7-a197-aa2e07f87202 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:28.491950 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 551/1106] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:28 2022] GET /identity/v3/role_assignments?user.id=4ab95a4201b447fd8f42098f1a010900&scope.project.id=0aed73c3c18141ec8d49a3aeaa87b7bd&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:30.692462 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752f462-9068-4d8b-a833-066e009d2469 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.693463 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752f462-9068-4d8b-a833-066e009d2469 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.693463 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752f462-9068-4d8b-a833-066e009d2469 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.694799 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 556/1107] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:30.700067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f6b4c-2ac2-4aeb-beef-bd4fb038cc34 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.700547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f6b4c-2ac2-4aeb-beef-bd4fb038cc34 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.700642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f6b4c-2ac2-4aeb-beef-bd4fb038cc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.718830 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0f1f6b4c-2ac2-4aeb-beef-bd4fb038cc34 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:30.720161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0f1f6b4c-2ac2-4aeb-beef-bd4fb038cc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:30.786765 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 552/1108] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:31.060651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752f462-9068-4d8b-a833-066e009d2469 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.088260 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:31.163113 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752f462-9068-4d8b-a833-066e009d2469 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:31.196789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a1e01-91e8-4044-87f5-5522d2e3ee3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:31.197402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a1e01-91e8-4044-87f5-5522d2e3ee3a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.197681 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a1e01-91e8-4044-87f5-5522d2e3ee3a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.197949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a1e01-91e8-4044-87f5-5522d2e3ee3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.199064 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a1e01-91e8-4044-87f5-5522d2e3ee3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:31.200503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a1e01-91e8-4044-87f5-5522d2e3ee3a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:31.202055 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 557/1109] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:31.207846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f6b4c-2ac2-4aeb-beef-bd4fb038cc34 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.222623 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:31.303111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f6b4c-2ac2-4aeb-beef-bd4fb038cc34 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:31.335988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5daaa0-af38-4bbd-9c80-feab11e7921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:31.336504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5daaa0-af38-4bbd-9c80-feab11e7921e None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.336777 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5daaa0-af38-4bbd-9c80-feab11e7921e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.337042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5daaa0-af38-4bbd-9c80-feab11e7921e None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.337680 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5daaa0-af38-4bbd-9c80-feab11e7921e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:31.339117 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5daaa0-af38-4bbd-9c80-feab11e7921e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:31.453068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 553/1110] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:39:31 2022] POST /identity/v3/users => generated 286 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:39:33.696292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1228168-f8e3-41d8-b561-38aad8bd2ae3 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:33.696830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1228168-f8e3-41d8-b561-38aad8bd2ae3 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:33.696830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1228168-f8e3-41d8-b561-38aad8bd2ae3 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.698184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 558/1111] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:33.704134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c114117-b369-4031-82ff-bb28eeb96462 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:33.704668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c114117-b369-4031-82ff-bb28eeb96462 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:33.704668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c114117-b369-4031-82ff-bb28eeb96462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.724790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-2c114117-b369-4031-82ff-bb28eeb96462 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:33.725987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2c114117-b369-4031-82ff-bb28eeb96462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:33.791916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:33.797536 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 554/1112] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:34.071135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1228168-f8e3-41d8-b561-38aad8bd2ae3 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:34.163407 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:34.173040 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1228168-f8e3-41d8-b561-38aad8bd2ae3 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:34.207219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9a9c5-ae03-481d-888c-6b720649ab99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:34.207838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9a9c5-ae03-481d-888c-6b720649ab99 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:34.208658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9a9c5-ae03-481d-888c-6b720649ab99 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:34.208931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9a9c5-ae03-481d-888c-6b720649ab99 None admin] PATH_INFO: `/v3/roles/1a62bc555eb345fc8679cfa677069fd7` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:34.210580 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9a9c5-ae03-481d-888c-6b720649ab99 None admin] RBAC: Authorizing `identity:get_role(role_id=1a62bc555eb345fc8679cfa677069fd7)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:34.211984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9a9c5-ae03-481d-888c-6b720649ab99 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:34.213314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 559/1113] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:34 2022] GET /identity/v3/roles/1a62bc555eb345fc8679cfa677069fd7 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:34.219167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c114117-b369-4031-82ff-bb28eeb96462 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:34.315848 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c114117-b369-4031-82ff-bb28eeb96462 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:34.349586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda02f1-7249-4ca3-8ea7-22d149ac66c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:34.350119 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda02f1-7249-4ca3-8ea7-22d149ac66c9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:34.350396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda02f1-7249-4ca3-8ea7-22d149ac66c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:34.350665 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda02f1-7249-4ca3-8ea7-22d149ac66c9 None admin] PATH_INFO: `/v3/users/14ecc8d9751448c1b29be24150bf2c55` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:34.363292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda02f1-7249-4ca3-8ea7-22d149ac66c9 None admin] RBAC: Authorizing `identity:get_user(user_id=14ecc8d9751448c1b29be24150bf2c55)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:34.364731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda02f1-7249-4ca3-8ea7-22d149ac66c9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:34.366335 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 555/1114] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:34 2022] GET /identity/v3/users/14ecc8d9751448c1b29be24150bf2c55 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:34.372231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9a9c5-ae03-481d-888c-6b720649ab99 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:34.471543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9a9c5-ae03-481d-888c-6b720649ab99 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:34.506484 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6453a-a30f-4d9d-9ec9-338176ff4a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:34.507136 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6453a-a30f-4d9d-9ec9-338176ff4a63 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:34.507395 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6453a-a30f-4d9d-9ec9-338176ff4a63 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:34.507668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6453a-a30f-4d9d-9ec9-338176ff4a63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:34.535278 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:34.574318 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6453a-a30f-4d9d-9ec9-338176ff4a63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:34.575806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6453a-a30f-4d9d-9ec9-338176ff4a63 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:34.671249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 560/1115] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:39:34.677946 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda02f1-7249-4ca3-8ea7-22d149ac66c9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:34.731797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:34.774369 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda02f1-7249-4ca3-8ea7-22d149ac66c9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:34.808062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2b147-796b-4c63-9a5f-43633bb36629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:34.808671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2b147-796b-4c63-9a5f-43633bb36629 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:34.808947 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2b147-796b-4c63-9a5f-43633bb36629 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:34.809217 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2b147-796b-4c63-9a5f-43633bb36629 None admin] PATH_INFO: `/v3/projects/0aed73c3c18141ec8d49a3aeaa87b7bd` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:34.810304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2b147-796b-4c63-9a5f-43633bb36629 None admin] RBAC: Authorizing `identity:get_project(project_id=0aed73c3c18141ec8d49a3aeaa87b7bd)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:34.811761 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2b147-796b-4c63-9a5f-43633bb36629 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:34.813364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 556/1116] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:34 2022] GET /identity/v3/projects/0aed73c3c18141ec8d49a3aeaa87b7bd => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:34.819445 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6453a-a30f-4d9d-9ec9-338176ff4a63 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:34.877328 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:34.921344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6453a-a30f-4d9d-9ec9-338176ff4a63 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:34.954914 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79323eb6-4f7c-4456-8543-03dff0dac843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:34.955532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79323eb6-4f7c-4456-8543-03dff0dac843 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:34.955836 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79323eb6-4f7c-4456-8543-03dff0dac843 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:34.956114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79323eb6-4f7c-4456-8543-03dff0dac843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:34.956831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79323eb6-4f7c-4456-8543-03dff0dac843 None admin] RBAC: Adding query filter params (role.id=1a62bc555eb345fc8679cfa677069fd7, scope.project.id=0aed73c3c18141ec8d49a3aeaa87b7bd, user.id=14ecc8d9751448c1b29be24150bf2c55) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:34.957143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79323eb6-4f7c-4456-8543-03dff0dac843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:34.959090 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79323eb6-4f7c-4456-8543-03dff0dac843 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:34.964774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 561/1117] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:34 2022] GET /identity/v3/role_assignments?user.id=14ecc8d9751448c1b29be24150bf2c55&scope.project.id=0aed73c3c18141ec8d49a3aeaa87b7bd&role.id=1a62bc555eb345fc8679cfa677069fd7 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:37.081800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756101a-a525-4f64-99f9-db4e77b82c83 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.082333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756101a-a525-4f64-99f9-db4e77b82c83 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.082333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756101a-a525-4f64-99f9-db4e77b82c83 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.084337 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 557/1118] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:37.089647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4838598-0b3f-4ea6-816d-00064aadf9e3 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.090054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4838598-0b3f-4ea6-816d-00064aadf9e3 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.090233 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4838598-0b3f-4ea6-816d-00064aadf9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.109151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d4838598-0b3f-4ea6-816d-00064aadf9e3 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:37.110335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d4838598-0b3f-4ea6-816d-00064aadf9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:37.178288 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 562/1119] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:37.450040 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756101a-a525-4f64-99f9-db4e77b82c83 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.550368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756101a-a525-4f64-99f9-db4e77b82c83 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.573370 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:37.583443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0aca04-4c05-48d0-a9ba-946b0c431e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.588786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0aca04-4c05-48d0-a9ba-946b0c431e1f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.588786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0aca04-4c05-48d0-a9ba-946b0c431e1f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.588786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0aca04-4c05-48d0-a9ba-946b0c431e1f None admin] PATH_INFO: `/v3/roles/1a62bc555eb345fc8679cfa677069fd7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.588786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0aca04-4c05-48d0-a9ba-946b0c431e1f None admin] RBAC: Authorizing `identity:get_role(role_id=1a62bc555eb345fc8679cfa677069fd7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:37.588786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0aca04-4c05-48d0-a9ba-946b0c431e1f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:37.589168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 558/1120] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:37 2022] GET /identity/v3/roles/1a62bc555eb345fc8679cfa677069fd7 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:37.595019 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4838598-0b3f-4ea6-816d-00064aadf9e3 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.649071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:37.692939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4838598-0b3f-4ea6-816d-00064aadf9e3 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.726411 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f1cd3-222e-4bf2-90aa-2dd637fe8c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.727010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f1cd3-222e-4bf2-90aa-2dd637fe8c16 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.727294 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f1cd3-222e-4bf2-90aa-2dd637fe8c16 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.727569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f1cd3-222e-4bf2-90aa-2dd637fe8c16 None admin] PATH_INFO: `/v3/users/14ecc8d9751448c1b29be24150bf2c55` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.728683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f1cd3-222e-4bf2-90aa-2dd637fe8c16 None admin] RBAC: Authorizing `identity:get_user(user_id=14ecc8d9751448c1b29be24150bf2c55)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:37.730118 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f1cd3-222e-4bf2-90aa-2dd637fe8c16 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:37.731734 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 563/1121] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:37 2022] GET /identity/v3/users/14ecc8d9751448c1b29be24150bf2c55 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:37.737519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0aca04-4c05-48d0-a9ba-946b0c431e1f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.837738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0aca04-4c05-48d0-a9ba-946b0c431e1f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.872372 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb721a83-ede8-41ba-a2b7-648372326436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.872982 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb721a83-ede8-41ba-a2b7-648372326436 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.873257 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb721a83-ede8-41ba-a2b7-648372326436 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.873526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb721a83-ede8-41ba-a2b7-648372326436 None admin] PATH_INFO: `/v3/projects/0aed73c3c18141ec8d49a3aeaa87b7bd` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.874623 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb721a83-ede8-41ba-a2b7-648372326436 None admin] RBAC: Authorizing `identity:get_project(project_id=0aed73c3c18141ec8d49a3aeaa87b7bd)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:37.876076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb721a83-ede8-41ba-a2b7-648372326436 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:37.877693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 559/1122] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:37 2022] GET /identity/v3/projects/0aed73c3c18141ec8d49a3aeaa87b7bd => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:37.883978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f1cd3-222e-4bf2-90aa-2dd637fe8c16 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.982083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f1cd3-222e-4bf2-90aa-2dd637fe8c16 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.004660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:38.015981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568a4cd-2b8c-4776-b89a-f6fbca4a6fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.016631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568a4cd-2b8c-4776-b89a-f6fbca4a6fef None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.016906 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568a4cd-2b8c-4776-b89a-f6fbca4a6fef None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.017174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568a4cd-2b8c-4776-b89a-f6fbca4a6fef None admin] PATH_INFO: `/v3/projects/0aed73c3c18141ec8d49a3aeaa87b7bd/users/14ecc8d9751448c1b29be24150bf2c55/roles/1a62bc555eb345fc8679cfa677069fd7` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.019324 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568a4cd-2b8c-4776-b89a-f6fbca4a6fef None admin] RBAC: Authorizing `identity:create_grant(project_id=0aed73c3c18141ec8d49a3aeaa87b7bd, user_id=14ecc8d9751448c1b29be24150bf2c55, role_id=1a62bc555eb345fc8679cfa677069fd7)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.026365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568a4cd-2b8c-4776-b89a-f6fbca4a6fef None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.127618 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 564/1123] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:39:37 2022] PUT /identity/v3/projects/0aed73c3c18141ec8d49a3aeaa87b7bd/users/14ecc8d9751448c1b29be24150bf2c55/roles/1a62bc555eb345fc8679cfa677069fd7 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:39:40.299949 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2084b-79f8-4c23-a9ba-0ed183c966b6 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.300600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2084b-79f8-4c23-a9ba-0ed183c966b6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.300600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c2084b-79f8-4c23-a9ba-0ed183c966b6 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.301816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 560/1124] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:40.307389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd067490-c561-4418-b9c9-52ca84219897 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.307823 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd067490-c561-4418-b9c9-52ca84219897 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.307936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd067490-c561-4418-b9c9-52ca84219897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.325567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-dd067490-c561-4418-b9c9-52ca84219897 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:40.326742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-dd067490-c561-4418-b9c9-52ca84219897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:40.392371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 565/1125] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:40.658192 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2084b-79f8-4c23-a9ba-0ed183c966b6 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:40.717425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:40.757217 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c2084b-79f8-4c23-a9ba-0ed183c966b6 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:40.789827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338a1a2-1794-4c32-819d-d6a6a9d0b5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:40.790434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338a1a2-1794-4c32-819d-d6a6a9d0b5d2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.790708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338a1a2-1794-4c32-819d-d6a6a9d0b5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.791345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338a1a2-1794-4c32-819d-d6a6a9d0b5d2 None admin] PATH_INFO: `/v3/roles/1a62bc555eb345fc8679cfa677069fd7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.792706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338a1a2-1794-4c32-819d-d6a6a9d0b5d2 None admin] RBAC: Authorizing `identity:get_role(role_id=1a62bc555eb345fc8679cfa677069fd7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:40.794062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338a1a2-1794-4c32-819d-d6a6a9d0b5d2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:40.795400 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 561/1126] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:40 2022] GET /identity/v3/roles/1a62bc555eb345fc8679cfa677069fd7 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:40.801340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd067490-c561-4418-b9c9-52ca84219897 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:40.898690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd067490-c561-4418-b9c9-52ca84219897 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:40.911544 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:40.932572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e09410-ed36-4350-9963-386add3caad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:40.933188 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e09410-ed36-4350-9963-386add3caad6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.933468 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e09410-ed36-4350-9963-386add3caad6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.933742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e09410-ed36-4350-9963-386add3caad6 None admin] PATH_INFO: `/v3/users/14ecc8d9751448c1b29be24150bf2c55` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.934856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e09410-ed36-4350-9963-386add3caad6 None admin] RBAC: Authorizing `identity:get_user(user_id=14ecc8d9751448c1b29be24150bf2c55)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:40.936315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e09410-ed36-4350-9963-386add3caad6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:40.937881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 566/1127] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:40 2022] GET /identity/v3/users/14ecc8d9751448c1b29be24150bf2c55 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:40.943672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338a1a2-1794-4c32-819d-d6a6a9d0b5d2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:41.040591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338a1a2-1794-4c32-819d-d6a6a9d0b5d2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:41.069949 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:41.077352 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0530b209-c5cb-464f-b128-e77e2296a077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:41.077941 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530b209-c5cb-464f-b128-e77e2296a077 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:41.078237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530b209-c5cb-464f-b128-e77e2296a077 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:41.078502 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530b209-c5cb-464f-b128-e77e2296a077 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:41.143693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0530b209-c5cb-464f-b128-e77e2296a077 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:41.145122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0530b209-c5cb-464f-b128-e77e2296a077 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:41.238914 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 562/1128] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:39:41.244997 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e09410-ed36-4350-9963-386add3caad6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:41.341907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e09410-ed36-4350-9963-386add3caad6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:41.374905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a879c-81e0-4695-8662-d415a2e46461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:41.375610 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a879c-81e0-4695-8662-d415a2e46461 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:41.375887 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a879c-81e0-4695-8662-d415a2e46461 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:41.376162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a879c-81e0-4695-8662-d415a2e46461 None admin] PATH_INFO: `/v3/projects/0aed73c3c18141ec8d49a3aeaa87b7bd` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:41.377270 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a879c-81e0-4695-8662-d415a2e46461 None admin] RBAC: Authorizing `identity:get_project(project_id=0aed73c3c18141ec8d49a3aeaa87b7bd)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:41.378723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a879c-81e0-4695-8662-d415a2e46461 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:41.380889 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 567/1129] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:41 2022] GET /identity/v3/projects/0aed73c3c18141ec8d49a3aeaa87b7bd => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:41.387139 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0530b209-c5cb-464f-b128-e77e2296a077 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:41.414718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:41.487742 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0530b209-c5cb-464f-b128-e77e2296a077 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:41.525541 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15602bd8-1058-4c36-87db-189c3e33f519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:41.525541 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15602bd8-1058-4c36-87db-189c3e33f519 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:41.525541 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15602bd8-1058-4c36-87db-189c3e33f519 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:41.525541 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15602bd8-1058-4c36-87db-189c3e33f519 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:41.525934 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15602bd8-1058-4c36-87db-189c3e33f519 None admin] RBAC: Adding query filter params (role.id=1a62bc555eb345fc8679cfa677069fd7, scope.project.id=0aed73c3c18141ec8d49a3aeaa87b7bd, user.id=14ecc8d9751448c1b29be24150bf2c55) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:41.525934 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15602bd8-1058-4c36-87db-189c3e33f519 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:41.525934 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15602bd8-1058-4c36-87db-189c3e33f519 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:41.530524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 563/1130] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:41 2022] GET /identity/v3/role_assignments?user.id=14ecc8d9751448c1b29be24150bf2c55&scope.project.id=0aed73c3c18141ec8d49a3aeaa87b7bd&role.id=1a62bc555eb345fc8679cfa677069fd7 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:43.683878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cabb8-860b-432b-848a-697f563e4f39 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.684400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cabb8-860b-432b-848a-697f563e4f39 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.684400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cabb8-860b-432b-848a-697f563e4f39 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.685727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 568/1131] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:43.691169 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed905ad-4783-4bd6-976f-27364673d268 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.691574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed905ad-4783-4bd6-976f-27364673d268 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.691711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed905ad-4783-4bd6-976f-27364673d268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.709696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7ed905ad-4783-4bd6-976f-27364673d268 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:43.711179 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7ed905ad-4783-4bd6-976f-27364673d268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:43.781981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 564/1132] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:44.060374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cabb8-860b-432b-848a-697f563e4f39 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.115384 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:44.172169 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cabb8-860b-432b-848a-697f563e4f39 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.209546 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a60cd1-3ae0-4111-ac9d-adc69a55aa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.210181 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a60cd1-3ae0-4111-ac9d-adc69a55aa07 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.210454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a60cd1-3ae0-4111-ac9d-adc69a55aa07 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.210721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a60cd1-3ae0-4111-ac9d-adc69a55aa07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.212138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a60cd1-3ae0-4111-ac9d-adc69a55aa07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:44.213625 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a60cd1-3ae0-4111-ac9d-adc69a55aa07 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.215227 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 569/1133] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:44.221360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed905ad-4783-4bd6-976f-27364673d268 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.245534 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:44.327824 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed905ad-4783-4bd6-976f-27364673d268 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.363137 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2038d-68c2-4463-885c-209edd6df1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.363738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2038d-68c2-4463-885c-209edd6df1c2 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.364017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2038d-68c2-4463-885c-209edd6df1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.364284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2038d-68c2-4463-885c-209edd6df1c2 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.364952 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2038d-68c2-4463-885c-209edd6df1c2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:44.366393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2038d-68c2-4463-885c-209edd6df1c2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.495595 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 565/1134] 10.222.0.91 () {60 vars in 1141 bytes} [Thu Sep 8 13:39:44 2022] POST /identity/v3/projects => generated 311 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:46.714364 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20640d9-3959-4de6-8e7c-310b2dcaa893 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:46.714982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20640d9-3959-4de6-8e7c-310b2dcaa893 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:46.714982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20640d9-3959-4de6-8e7c-310b2dcaa893 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:46.716267 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 570/1135] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:46.721700 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef667df3-37a2-41d4-bc28-a04dad66c370 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:46.722138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef667df3-37a2-41d4-bc28-a04dad66c370 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:46.722233 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef667df3-37a2-41d4-bc28-a04dad66c370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:46.741509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ef667df3-37a2-41d4-bc28-a04dad66c370 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:46.742596 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ef667df3-37a2-41d4-bc28-a04dad66c370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:46.810905 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 566/1136] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:47.082099 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20640d9-3959-4de6-8e7c-310b2dcaa893 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:47.195187 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20640d9-3959-4de6-8e7c-310b2dcaa893 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:47.219193 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:47.236583 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e6c980-2196-45ae-9c3c-acef08b295d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:47.237418 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6c980-2196-45ae-9c3c-acef08b295d6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:47.238084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6c980-2196-45ae-9c3c-acef08b295d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:47.238084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6c980-2196-45ae-9c3c-acef08b295d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:47.239032 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6c980-2196-45ae-9c3c-acef08b295d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:47.241051 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e6c980-2196-45ae-9c3c-acef08b295d6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:47.242649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 571/1137] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:47.248578 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef667df3-37a2-41d4-bc28-a04dad66c370 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:47.251791 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:47.354536 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef667df3-37a2-41d4-bc28-a04dad66c370 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:47.393181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b282914-c833-41ba-bd37-a8fff867fd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:47.393779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b282914-c833-41ba-bd37-a8fff867fd95 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:47.394054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b282914-c833-41ba-bd37-a8fff867fd95 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:47.394321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b282914-c833-41ba-bd37-a8fff867fd95 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:47.394992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b282914-c833-41ba-bd37-a8fff867fd95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:47.396961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b282914-c833-41ba-bd37-a8fff867fd95 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:47.523743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 567/1138] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:39:47 2022] POST /identity/v3/users => generated 286 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:49.828225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c9943-fd3c-4ff2-80c3-8f8b48191099 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:49.828774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c9943-fd3c-4ff2-80c3-8f8b48191099 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:49.829087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c9943-fd3c-4ff2-80c3-8f8b48191099 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:49.831594 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 572/1139] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:49.837479 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3388d-f289-46c3-94ca-48602c945582 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:49.838004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3388d-f289-46c3-94ca-48602c945582 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:49.838004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad3388d-f289-46c3-94ca-48602c945582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:49.857582 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-2ad3388d-f289-46c3-94ca-48602c945582 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:49.858804 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2ad3388d-f289-46c3-94ca-48602c945582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:49.933961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:49.946283 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 568/1140] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:50.251992 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c9943-fd3c-4ff2-80c3-8f8b48191099 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:50.374395 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c9943-fd3c-4ff2-80c3-8f8b48191099 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:50.416238 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce54eae-fdbf-44e9-8579-269d150d9f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:50.419342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce54eae-fdbf-44e9-8579-269d150d9f41 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:50.419342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce54eae-fdbf-44e9-8579-269d150d9f41 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:50.419342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce54eae-fdbf-44e9-8579-269d150d9f41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:50.434344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce54eae-fdbf-44e9-8579-269d150d9f41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:50.439347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-1ce54eae-fdbf-44e9-8579-269d150d9f41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:50.439347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:50.439347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:50.439347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:50.439347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:50.439347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:50.439347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:50.439347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:50.439347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:50.439347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:50.439347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:50.439347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:50.439347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:50.439347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:50.439347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:50.439347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:50.440596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:50.440596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:50.440596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:50.440596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:50.440596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:50.440596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:50.440596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:50.440596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:50.440596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:50.440596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:50.440596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:50.440596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:50.440596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:50.440596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:50.440596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:50.440596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:50.441786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:50.441786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:50.441786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:50.441786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:50.441786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:50.441786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:50.441786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:50.441786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:50.441786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:50.441786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:50.441786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:50.441786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:50.441786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 573/1141] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:50.449502 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3388d-f289-46c3-94ca-48602c945582 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:50.594177 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad3388d-f289-46c3-94ca-48602c945582 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:50.630377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710a329-21a3-4e3e-88ea-fc805e4ff3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:50.631129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710a329-21a3-4e3e-88ea-fc805e4ff3fc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:50.631453 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710a329-21a3-4e3e-88ea-fc805e4ff3fc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:50.631767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710a329-21a3-4e3e-88ea-fc805e4ff3fc None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:50.632478 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710a329-21a3-4e3e-88ea-fc805e4ff3fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:50.632858 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710a329-21a3-4e3e-88ea-fc805e4ff3fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:50.634292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710a329-21a3-4e3e-88ea-fc805e4ff3fc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:50.643117 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 569/1142] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:50.649793 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce54eae-fdbf-44e9-8579-269d150d9f41 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:50.701559 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:50.780646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce54eae-fdbf-44e9-8579-269d150d9f41 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:50.816697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41630cda-07b5-4700-92c5-48e94212cf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:50.817318 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41630cda-07b5-4700-92c5-48e94212cf33 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:50.817590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41630cda-07b5-4700-92c5-48e94212cf33 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:50.817994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41630cda-07b5-4700-92c5-48e94212cf33 None admin] PATH_INFO: `/v3/users/264bbb737b2c43c98e4e48ba392f78fb` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:50.834363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41630cda-07b5-4700-92c5-48e94212cf33 None admin] RBAC: Authorizing `identity:get_user(user_id=264bbb737b2c43c98e4e48ba392f78fb)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:50.835871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41630cda-07b5-4700-92c5-48e94212cf33 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:50.837797 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 574/1143] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:50 2022] GET /identity/v3/users/264bbb737b2c43c98e4e48ba392f78fb => generated 286 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:50.844475 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710a329-21a3-4e3e-88ea-fc805e4ff3fc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:50.899974 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:50.949775 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710a329-21a3-4e3e-88ea-fc805e4ff3fc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:50.985318 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4227f8f-ca5c-45c7-818e-c787f8df4d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:50.985920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4227f8f-ca5c-45c7-818e-c787f8df4d25 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:50.986196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4227f8f-ca5c-45c7-818e-c787f8df4d25 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:50.986466 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4227f8f-ca5c-45c7-818e-c787f8df4d25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.062204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4227f8f-ca5c-45c7-818e-c787f8df4d25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:51.063699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4227f8f-ca5c-45c7-818e-c787f8df4d25 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:51.126940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:51.180017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 570/1144] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:39:51.189315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41630cda-07b5-4700-92c5-48e94212cf33 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:51.348310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41630cda-07b5-4700-92c5-48e94212cf33 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:51.351353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:51.395296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8916f7-183e-4369-b932-bb151effe85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:51.395923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8916f7-183e-4369-b932-bb151effe85e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.396193 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8916f7-183e-4369-b932-bb151effe85e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.397067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8916f7-183e-4369-b932-bb151effe85e None admin] PATH_INFO: `/v3/projects/1ed09a80e6804721ab026bdc9ed6fc6b` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.398369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8916f7-183e-4369-b932-bb151effe85e None admin] RBAC: Authorizing `identity:get_project(project_id=1ed09a80e6804721ab026bdc9ed6fc6b)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:51.399938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8916f7-183e-4369-b932-bb151effe85e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:51.401820 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 575/1145] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:51 2022] GET /identity/v3/projects/1ed09a80e6804721ab026bdc9ed6fc6b => generated 311 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:51.409285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4227f8f-ca5c-45c7-818e-c787f8df4d25 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:51.547343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4227f8f-ca5c-45c7-818e-c787f8df4d25 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:51.591640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecbcc3-cf26-4f5e-b612-912d44936f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:51.592265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecbcc3-cf26-4f5e-b612-912d44936f9f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.592544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecbcc3-cf26-4f5e-b612-912d44936f9f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.592813 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecbcc3-cf26-4f5e-b612-912d44936f9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.593568 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ecbcc3-cf26-4f5e-b612-912d44936f9f None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=1ed09a80e6804721ab026bdc9ed6fc6b, user.id=264bbb737b2c43c98e4e48ba392f78fb) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:51.593873 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ecbcc3-cf26-4f5e-b612-912d44936f9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:51.595378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ecbcc3-cf26-4f5e-b612-912d44936f9f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:51.601669 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 571/1146] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:51 2022] GET /identity/v3/role_assignments?user.id=264bbb737b2c43c98e4e48ba392f78fb&scope.project.id=1ed09a80e6804721ab026bdc9ed6fc6b&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:53.776421 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8026ac3-c75e-452a-bc05-867f77daa1ed None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:53.776943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8026ac3-c75e-452a-bc05-867f77daa1ed None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:53.776943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8026ac3-c75e-452a-bc05-867f77daa1ed None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:53.778228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 576/1147] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:53.783548 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec27471-61c9-491d-816d-f24b9c8ef770 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:53.784097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec27471-61c9-491d-816d-f24b9c8ef770 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:53.784097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec27471-61c9-491d-816d-f24b9c8ef770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:53.804488 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4ec27471-61c9-491d-816d-f24b9c8ef770 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:53.805671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4ec27471-61c9-491d-816d-f24b9c8ef770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:53.818052 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:53.878177 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 572/1148] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:54.161817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8026ac3-c75e-452a-bc05-867f77daa1ed None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:54.276322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8026ac3-c75e-452a-bc05-867f77daa1ed None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:54.310302 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841f8ec-2679-462c-8fe2-405372ed1139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:54.310949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841f8ec-2679-462c-8fe2-405372ed1139 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.311227 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841f8ec-2679-462c-8fe2-405372ed1139 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.311494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841f8ec-2679-462c-8fe2-405372ed1139 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.322063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841f8ec-2679-462c-8fe2-405372ed1139 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:54.326805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-7841f8ec-2679-462c-8fe2-405372ed1139 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:54.326805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:54.326805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:54.326805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:54.326805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:54.326805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:54.326805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:54.326805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:54.326805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:54.326805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:54.326805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:54.326805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:54.326805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:54.326805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:54.326805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:54.326805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:54.327767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:54.327767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:54.327767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:54.327767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:54.327767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:54.327767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:54.327767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:54.327767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:54.327767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:54.327767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:54.327767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:54.327767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:54.327767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:54.327767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:54.327767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:54.327767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:54.328778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:54.328778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:54.328778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:54.328778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:54.328778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:54.328778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:54.328778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:54.328778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:54.328778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:54.328778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:54.328778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:54.328778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:39:54.328778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 577/1149] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:54.335198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec27471-61c9-491d-816d-f24b9c8ef770 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:54.436198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec27471-61c9-491d-816d-f24b9c8ef770 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:54.470601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4da770-c49c-4986-9a69-c27db211b041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:54.471235 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4da770-c49c-4986-9a69-c27db211b041 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.471509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4da770-c49c-4986-9a69-c27db211b041 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.471776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4da770-c49c-4986-9a69-c27db211b041 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.472431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4da770-c49c-4986-9a69-c27db211b041 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:54.472727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4da770-c49c-4986-9a69-c27db211b041 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:54.474121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4da770-c49c-4986-9a69-c27db211b041 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:54.483527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 573/1150] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:54.489322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841f8ec-2679-462c-8fe2-405372ed1139 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:54.519795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:54.593158 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841f8ec-2679-462c-8fe2-405372ed1139 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:54.628543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7d4d9-e0af-4095-b786-d87996cb0a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:54.629165 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7d4d9-e0af-4095-b786-d87996cb0a62 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.629437 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7d4d9-e0af-4095-b786-d87996cb0a62 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.629731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a7d4d9-e0af-4095-b786-d87996cb0a62 None admin] PATH_INFO: `/v3/users/264bbb737b2c43c98e4e48ba392f78fb` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.630842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7d4d9-e0af-4095-b786-d87996cb0a62 None admin] RBAC: Authorizing `identity:get_user(user_id=264bbb737b2c43c98e4e48ba392f78fb)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:54.632304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a7d4d9-e0af-4095-b786-d87996cb0a62 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:54.633895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 578/1151] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:54 2022] GET /identity/v3/users/264bbb737b2c43c98e4e48ba392f78fb => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:54.639863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4da770-c49c-4986-9a69-c27db211b041 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:54.652045 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:54.756921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4da770-c49c-4986-9a69-c27db211b041 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:54.796440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ef015-faeb-434e-addf-6659d987dfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:54.797866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ef015-faeb-434e-addf-6659d987dfd0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.797866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ef015-faeb-434e-addf-6659d987dfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.797866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ef015-faeb-434e-addf-6659d987dfd0 None admin] PATH_INFO: `/v3/projects/1ed09a80e6804721ab026bdc9ed6fc6b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.798828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ef015-faeb-434e-addf-6659d987dfd0 None admin] RBAC: Authorizing `identity:get_project(project_id=1ed09a80e6804721ab026bdc9ed6fc6b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:54.800282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ef015-faeb-434e-addf-6659d987dfd0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:54.801912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 574/1152] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:54 2022] GET /identity/v3/projects/1ed09a80e6804721ab026bdc9ed6fc6b => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:54.808275 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7d4d9-e0af-4095-b786-d87996cb0a62 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:54.909807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:54.913638 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a7d4d9-e0af-4095-b786-d87996cb0a62 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:54.948057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353effa-1aba-462f-b79f-fb8ab6c6f30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:54.948713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353effa-1aba-462f-b79f-fb8ab6c6f30e None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.948993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353effa-1aba-462f-b79f-fb8ab6c6f30e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.949267 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353effa-1aba-462f-b79f-fb8ab6c6f30e None admin] PATH_INFO: `/v3/projects/1ed09a80e6804721ab026bdc9ed6fc6b/users/264bbb737b2c43c98e4e48ba392f78fb/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.951466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353effa-1aba-462f-b79f-fb8ab6c6f30e None admin] RBAC: Authorizing `identity:create_grant(project_id=1ed09a80e6804721ab026bdc9ed6fc6b, user_id=264bbb737b2c43c98e4e48ba392f78fb, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:54.953081 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353effa-1aba-462f-b79f-fb8ab6c6f30e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:55.044109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 579/1153] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:39:54 2022] PUT /identity/v3/projects/1ed09a80e6804721ab026bdc9ed6fc6b/users/264bbb737b2c43c98e4e48ba392f78fb/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:39:57.194319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d764a3-2f89-44d8-b3c5-37a6319ef7d5 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:57.194990 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d764a3-2f89-44d8-b3c5-37a6319ef7d5 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:57.194990 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d764a3-2f89-44d8-b3c5-37a6319ef7d5 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:57.196228 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 575/1154] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:57.202092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46efd3c-91d2-4ce0-b513-00835ffb435e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:57.202821 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46efd3c-91d2-4ce0-b513-00835ffb435e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:57.202821 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46efd3c-91d2-4ce0-b513-00835ffb435e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:57.229035 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e46efd3c-91d2-4ce0-b513-00835ffb435e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:57.230633 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e46efd3c-91d2-4ce0-b513-00835ffb435e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:57.306156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 580/1155] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:57.594261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d764a3-2f89-44d8-b3c5-37a6319ef7d5 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:57.677779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:57.703503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d764a3-2f89-44d8-b3c5-37a6319ef7d5 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:57.739260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de52d6-650b-4f06-907e-74d119a22a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:57.739765 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de52d6-650b-4f06-907e-74d119a22a70 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:57.740039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de52d6-650b-4f06-907e-74d119a22a70 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:57.740305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de52d6-650b-4f06-907e-74d119a22a70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:57.751743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7de52d6-650b-4f06-907e-74d119a22a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:57.756620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-d7de52d6-650b-4f06-907e-74d119a22a70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:57.756620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:57.756620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:57.756620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:57.756620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:57.756620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:57.756620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:57.756620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:57.756620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:57.756620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:57.756620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:57.756620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:57.756620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:57.756620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:57.756620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:57.756620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:57.757944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:57.757944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:57.757944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:57.757944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:57.757944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:57.757944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:57.757944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:57.757944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:57.757944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:57.757944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:57.757944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:57.757944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:57.757944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:57.757944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:57.757944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:57.757944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:57.759049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:57.759049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:57.759049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:57.759049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:57.759049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:57.759049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:57.759049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:57.759049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:57.759049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:57.759049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:57.759049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:57.759049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:39:57.759049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 576/1156] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:57.764335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46efd3c-91d2-4ce0-b513-00835ffb435e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:57.860912 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:57.873824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46efd3c-91d2-4ce0-b513-00835ffb435e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:57.909308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0a50d-ff09-46a8-8304-2b3e23212f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:57.909911 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0a50d-ff09-46a8-8304-2b3e23212f13 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:57.910190 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0a50d-ff09-46a8-8304-2b3e23212f13 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:57.910487 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0a50d-ff09-46a8-8304-2b3e23212f13 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:57.911342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0a50d-ff09-46a8-8304-2b3e23212f13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:57.911644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0a50d-ff09-46a8-8304-2b3e23212f13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:57.913548 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0a50d-ff09-46a8-8304-2b3e23212f13 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:57.921884 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 581/1157] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:57.928862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de52d6-650b-4f06-907e-74d119a22a70 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:58.030341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de52d6-650b-4f06-907e-74d119a22a70 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:58.067301 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236febb-893b-4680-a5a9-5c68611259be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:58.067919 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236febb-893b-4680-a5a9-5c68611259be None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:58.068192 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236febb-893b-4680-a5a9-5c68611259be None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:58.068461 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236febb-893b-4680-a5a9-5c68611259be None admin] PATH_INFO: `/v3/users/264bbb737b2c43c98e4e48ba392f78fb` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:58.070108 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236febb-893b-4680-a5a9-5c68611259be None admin] RBAC: Authorizing `identity:get_user(user_id=264bbb737b2c43c98e4e48ba392f78fb)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:58.071565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236febb-893b-4680-a5a9-5c68611259be None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:58.073147 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 577/1158] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:57 2022] GET /identity/v3/users/264bbb737b2c43c98e4e48ba392f78fb => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:58.079233 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0a50d-ff09-46a8-8304-2b3e23212f13 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:58.186744 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0a50d-ff09-46a8-8304-2b3e23212f13 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:58.222266 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72158604-b010-420e-b2cc-9603ef9b43f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:58.222984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72158604-b010-420e-b2cc-9603ef9b43f1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:58.223146 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72158604-b010-420e-b2cc-9603ef9b43f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:58.223413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72158604-b010-420e-b2cc-9603ef9b43f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:58.242375 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:58.292133 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72158604-b010-420e-b2cc-9603ef9b43f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:58.293495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72158604-b010-420e-b2cc-9603ef9b43f1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:58.398400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 582/1159] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:39:58.404494 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236febb-893b-4680-a5a9-5c68611259be None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:58.408713 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:58.538521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236febb-893b-4680-a5a9-5c68611259be None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:58.575112 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee40d4e-b477-48b2-809f-2e7ffb9db3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:58.575675 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee40d4e-b477-48b2-809f-2e7ffb9db3c0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:58.575987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee40d4e-b477-48b2-809f-2e7ffb9db3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:58.576262 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee40d4e-b477-48b2-809f-2e7ffb9db3c0 None admin] PATH_INFO: `/v3/projects/1ed09a80e6804721ab026bdc9ed6fc6b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:58.577422 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee40d4e-b477-48b2-809f-2e7ffb9db3c0 None admin] RBAC: Authorizing `identity:get_project(project_id=1ed09a80e6804721ab026bdc9ed6fc6b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:58.578868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee40d4e-b477-48b2-809f-2e7ffb9db3c0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:58.580682 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 578/1160] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:58 2022] GET /identity/v3/projects/1ed09a80e6804721ab026bdc9ed6fc6b => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:58.587539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72158604-b010-420e-b2cc-9603ef9b43f1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:58.643755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:58.702992 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72158604-b010-420e-b2cc-9603ef9b43f1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:58.738836 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f1163-be0f-4be9-8952-f1e1d33ec403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:58.739463 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f1163-be0f-4be9-8952-f1e1d33ec403 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:58.739739 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f1163-be0f-4be9-8952-f1e1d33ec403 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:58.740027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f1163-be0f-4be9-8952-f1e1d33ec403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:58.741359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f1163-be0f-4be9-8952-f1e1d33ec403 None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=1ed09a80e6804721ab026bdc9ed6fc6b, user.id=264bbb737b2c43c98e4e48ba392f78fb) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:58.741619 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f1163-be0f-4be9-8952-f1e1d33ec403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:58.743077 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f1163-be0f-4be9-8952-f1e1d33ec403 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:58.748894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 583/1161] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:58 2022] GET /identity/v3/role_assignments?user.id=264bbb737b2c43c98e4e48ba392f78fb&scope.project.id=1ed09a80e6804721ab026bdc9ed6fc6b&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:00.948426 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a58595-4cae-4197-9cf9-802c05bada1e None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.948982 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a58595-4cae-4197-9cf9-802c05bada1e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.948982 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a58595-4cae-4197-9cf9-802c05bada1e None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.950410 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 579/1162] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:00.956176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae037690-c15d-4772-9f3d-d3f9005107da None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.956643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae037690-c15d-4772-9f3d-d3f9005107da None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.956745 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae037690-c15d-4772-9f3d-d3f9005107da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.984514 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ae037690-c15d-4772-9f3d-d3f9005107da None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:00.985733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ae037690-c15d-4772-9f3d-d3f9005107da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:01.062765 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 584/1163] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:01.331184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a58595-4cae-4197-9cf9-802c05bada1e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:01.404068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:01.438238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a58595-4cae-4197-9cf9-802c05bada1e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:01.479218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185b36b-3884-4bb0-aa27-3ec5d7217d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.479919 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185b36b-3884-4bb0-aa27-3ec5d7217d6f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.480080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185b36b-3884-4bb0-aa27-3ec5d7217d6f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.480349 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185b36b-3884-4bb0-aa27-3ec5d7217d6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.559542 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185b36b-3884-4bb0-aa27-3ec5d7217d6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.561007 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185b36b-3884-4bb0-aa27-3ec5d7217d6f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:01.629577 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:01.659664 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 580/1164] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:01.667260 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae037690-c15d-4772-9f3d-d3f9005107da None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:01.713568 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:01.767906 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae037690-c15d-4772-9f3d-d3f9005107da None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:01.801606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57dc56-b935-4cb1-811d-18f6f216ddbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.802221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57dc56-b935-4cb1-811d-18f6f216ddbd None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.802499 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57dc56-b935-4cb1-811d-18f6f216ddbd None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.802765 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57dc56-b935-4cb1-811d-18f6f216ddbd None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.809883 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57dc56-b935-4cb1-811d-18f6f216ddbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.811353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57dc56-b935-4cb1-811d-18f6f216ddbd None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-5d57dc56-b935-4cb1-811d-18f6f216ddbd None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:01.823145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:01.824338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:01.824338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:01.824338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:01.824338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:01.824338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:01.824338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:01.824338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:01.824338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:01.824338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:01.824338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:01.824338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:01.824338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:01.824338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:01.824338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:01.824338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:01.824338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:01.825186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:01.825186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:01.825186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:01.825186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:01.825186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:01.825186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:01.825186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:01.825186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:01.825186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:01.825186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:01.825186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 08 13:40:01.825186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:01.825186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 08 13:40:01.825186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:40:01.825186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 585/1165] 10.222.0.91 () {56 vars in 1117 bytes} [Thu Sep 8 13:40:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:01.830817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185b36b-3884-4bb0-aa27-3ec5d7217d6f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:01.931430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185b36b-3884-4bb0-aa27-3ec5d7217d6f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:01.972346 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94539f4b-d836-4f51-a524-cd973d5b114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.973030 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94539f4b-d836-4f51-a524-cd973d5b114e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.973212 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94539f4b-d836-4f51-a524-cd973d5b114e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.973484 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94539f4b-d836-4f51-a524-cd973d5b114e None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.974117 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94539f4b-d836-4f51-a524-cd973d5b114e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:01.974420 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94539f4b-d836-4f51-a524-cd973d5b114e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.975832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94539f4b-d836-4f51-a524-cd973d5b114e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:01.982973 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 581/1166] 10.222.0.91 () {56 vars in 1115 bytes} [Thu Sep 8 13:40:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:04.143324 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fe4fe-b6d9-4496-89d9-a10d67ceb65e None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.143874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fe4fe-b6d9-4496-89d9-a10d67ceb65e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.143874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fe4fe-b6d9-4496-89d9-a10d67ceb65e None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.145217 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 586/1167] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:04.151298 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab5b75-bcf5-4d8f-8dcc-0229922bdbbf None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.152014 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab5b75-bcf5-4d8f-8dcc-0229922bdbbf None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.152014 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab5b75-bcf5-4d8f-8dcc-0229922bdbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.170293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0dab5b75-bcf5-4d8f-8dcc-0229922bdbbf None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:04.171602 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0dab5b75-bcf5-4d8f-8dcc-0229922bdbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:04.202589 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:04.242297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 582/1168] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:04.513109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fe4fe-b6d9-4496-89d9-a10d67ceb65e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:04.629942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fe4fe-b6d9-4496-89d9-a10d67ceb65e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:04.632430 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:04.669409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6b6db-90e6-4f56-b734-290ef14718c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:04.670008 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6b6db-90e6-4f56-b734-290ef14718c6 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.670289 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6b6db-90e6-4f56-b734-290ef14718c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.670561 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6b6db-90e6-4f56-b734-290ef14718c6 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.671325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6b6db-90e6-4f56-b734-290ef14718c6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:04.673318 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6b6db-90e6-4f56-b734-290ef14718c6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:04.754204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 587/1169] 10.222.0.91 () {60 vars in 1138 bytes} [Thu Sep 8 13:40:04 2022] POST /identity/v3/domains => generated 267 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:40:06.946737 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b4ab3-5554-46ae-b164-cada898ad084 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:06.947314 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b4ab3-5554-46ae-b164-cada898ad084 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:06.947314 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b4ab3-5554-46ae-b164-cada898ad084 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:06.948658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 583/1170] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:06.953931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc3a67-505f-487e-bf6b-d99b215afdc7 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:06.954384 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc3a67-505f-487e-bf6b-d99b215afdc7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:06.954476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc3a67-505f-487e-bf6b-d99b215afdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:06.975140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-5bdc3a67-505f-487e-bf6b-d99b215afdc7 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:06.976410 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5bdc3a67-505f-487e-bf6b-d99b215afdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:07.045561 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 588/1171] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:07.316590 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b4ab3-5554-46ae-b164-cada898ad084 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:07.425626 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b4ab3-5554-46ae-b164-cada898ad084 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:07.461673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d916c8-fbe9-4cb0-9ff3-f15d57decb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:07.462192 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d916c8-fbe9-4cb0-9ff3-f15d57decb19 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:07.462733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d916c8-fbe9-4cb0-9ff3-f15d57decb19 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:07.463122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d916c8-fbe9-4cb0-9ff3-f15d57decb19 None admin] PATH_INFO: `/v3/domains/1717fb0acdf3421bb8e423fac23c7b24` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:07.473759 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d916c8-fbe9-4cb0-9ff3-f15d57decb19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1717fb0acdf3421bb8e423fac23c7b24)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:07.475889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d916c8-fbe9-4cb0-9ff3-f15d57decb19 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:07.477558 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 584/1172] 10.222.0.91 () {56 vars in 1183 bytes} [Thu Sep 8 13:40:07 2022] GET /identity/v3/domains/1717fb0acdf3421bb8e423fac23c7b24 => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:07.483491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc3a67-505f-487e-bf6b-d99b215afdc7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:07.579217 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:07.588603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc3a67-505f-487e-bf6b-d99b215afdc7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:07.624465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaada5b-ce6c-4226-9a32-31b5c741ec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:07.625203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaada5b-ce6c-4226-9a32-31b5c741ec40 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:07.625481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaada5b-ce6c-4226-9a32-31b5c741ec40 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:07.625752 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaada5b-ce6c-4226-9a32-31b5c741ec40 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:07.626393 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaada5b-ce6c-4226-9a32-31b5c741ec40 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:07.627855 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaada5b-ce6c-4226-9a32-31b5c741ec40 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:07.857253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 589/1173] 10.222.0.91 () {60 vars in 1141 bytes} [Thu Sep 8 13:40:07 2022] POST /identity/v3/projects => generated 361 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:10.054455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66178b-06dd-4cd3-9075-dd60654b05d4 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:10.055023 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66178b-06dd-4cd3-9075-dd60654b05d4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:10.055622 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66178b-06dd-4cd3-9075-dd60654b05d4 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:10.056989 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 585/1174] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:40:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:10.062370 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7178476-2703-41b0-9ddb-b7842778a243 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:10.062871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7178476-2703-41b0-9ddb-b7842778a243 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:10.062871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7178476-2703-41b0-9ddb-b7842778a243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:10.081194 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b7178476-2703-41b0-9ddb-b7842778a243 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:10.082383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b7178476-2703-41b0-9ddb-b7842778a243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:10.149433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 590/1175] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:10.419718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66178b-06dd-4cd3-9075-dd60654b05d4 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:10.446762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:10.525191 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66178b-06dd-4cd3-9075-dd60654b05d4 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:10.560084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478be6c6-fdb4-4a73-8f00-2b2f79f61169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:10.560695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478be6c6-fdb4-4a73-8f00-2b2f79f61169 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:10.560967 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478be6c6-fdb4-4a73-8f00-2b2f79f61169 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:10.561234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478be6c6-fdb4-4a73-8f00-2b2f79f61169 None admin] PATH_INFO: `/v3/domains/1717fb0acdf3421bb8e423fac23c7b24` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:10.562325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478be6c6-fdb4-4a73-8f00-2b2f79f61169 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1717fb0acdf3421bb8e423fac23c7b24)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:10.563816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478be6c6-fdb4-4a73-8f00-2b2f79f61169 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:10.565396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 586/1176] 10.222.0.91 () {56 vars in 1183 bytes} [Thu Sep 8 13:40:10 2022] GET /identity/v3/domains/1717fb0acdf3421bb8e423fac23c7b24 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:10.571776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7178476-2703-41b0-9ddb-b7842778a243 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:10.649938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:10.672475 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7178476-2703-41b0-9ddb-b7842778a243 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:10.719311 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a68b5d-f3a9-47a5-813f-b7bdaf3dde25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:10.719930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a68b5d-f3a9-47a5-813f-b7bdaf3dde25 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:10.720205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a68b5d-f3a9-47a5-813f-b7bdaf3dde25 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:10.720476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a68b5d-f3a9-47a5-813f-b7bdaf3dde25 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:10.721137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a68b5d-f3a9-47a5-813f-b7bdaf3dde25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:10.722642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a68b5d-f3a9-47a5-813f-b7bdaf3dde25 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:10.856459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 591/1177] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:40:10 2022] POST /identity/v3/users => generated 311 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:13.079596 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f60eb-2720-49e7-bcad-68d7a1647de7 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:13.080133 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f60eb-2720-49e7-bcad-68d7a1647de7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:13.080133 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f60eb-2720-49e7-bcad-68d7a1647de7 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:13.081693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 587/1178] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:13.088173 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3b95c-cb8f-45c9-8ef2-761090aaa6b8 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:13.088604 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3b95c-cb8f-45c9-8ef2-761090aaa6b8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:13.088719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3b95c-cb8f-45c9-8ef2-761090aaa6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:13.107215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b1e3b95c-cb8f-45c9-8ef2-761090aaa6b8 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:13.108416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b1e3b95c-cb8f-45c9-8ef2-761090aaa6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:13.174748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 592/1179] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:13.445394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f60eb-2720-49e7-bcad-68d7a1647de7 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:13.604282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:13.617051 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f60eb-2720-49e7-bcad-68d7a1647de7 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:13.670093 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9e94e-0062-4471-b930-7608a0fe8eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:13.670725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9e94e-0062-4471-b930-7608a0fe8eb4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:13.671053 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9e94e-0062-4471-b930-7608a0fe8eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:13.671321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c9e94e-0062-4471-b930-7608a0fe8eb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:13.681998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c9e94e-0062-4471-b930-7608a0fe8eb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:13.687267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-76c9e94e-0062-4471-b930-7608a0fe8eb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:13.687267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:13.687267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:13.687267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:13.687267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:13.687267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:13.687267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:13.687267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:13.687267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:13.687267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:13.687267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:13.687267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:13.687267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:13.687267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:13.687267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:13.687267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:13.688574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:13.688574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:13.688574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:13.688574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:13.688574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:13.688574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:13.688574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:13.688574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:13.688574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:13.688574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:13.688574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:13.688574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:13.688574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:13.688574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:13.688574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:13.688574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:13.689444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:13.689444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:13.689444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:13.689444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:13.689444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:13.689444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:13.689444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:13.689444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:13.689444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:13.689444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:13.689444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:13.689444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:40:13.689444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 588/1180] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:13.695570 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3b95c-cb8f-45c9-8ef2-761090aaa6b8 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:13.758608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:13.800742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3b95c-cb8f-45c9-8ef2-761090aaa6b8 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:13.836947 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee1f52-1935-4d09-a296-7b60cb8cdad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:13.837612 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee1f52-1935-4d09-a296-7b60cb8cdad0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:13.838541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee1f52-1935-4d09-a296-7b60cb8cdad0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:13.838815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee1f52-1935-4d09-a296-7b60cb8cdad0 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:13.839599 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee1f52-1935-4d09-a296-7b60cb8cdad0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:13.840377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee1f52-1935-4d09-a296-7b60cb8cdad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:13.841777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee1f52-1935-4d09-a296-7b60cb8cdad0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:13.850272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 593/1181] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:13.856765 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9e94e-0062-4471-b930-7608a0fe8eb4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:13.957136 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c9e94e-0062-4471-b930-7608a0fe8eb4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:13.992153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deea6a5-182f-444c-86d1-44540f28aac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:13.992766 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deea6a5-182f-444c-86d1-44540f28aac5 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:13.993041 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deea6a5-182f-444c-86d1-44540f28aac5 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:13.993311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deea6a5-182f-444c-86d1-44540f28aac5 None admin] PATH_INFO: `/v3/users/e9091031eb324df0aad3f051ce7fdee6` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:14.006210 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deea6a5-182f-444c-86d1-44540f28aac5 None admin] RBAC: Authorizing `identity:get_user(user_id=e9091031eb324df0aad3f051ce7fdee6)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:14.007726 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deea6a5-182f-444c-86d1-44540f28aac5 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:14.009338 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 589/1182] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:13 2022] GET /identity/v3/users/e9091031eb324df0aad3f051ce7fdee6 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:14.015492 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee1f52-1935-4d09-a296-7b60cb8cdad0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:14.116976 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee1f52-1935-4d09-a296-7b60cb8cdad0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:14.139832 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:14.152186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae3d21-c7d1-42b4-b06d-aeeda703ca35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:14.152793 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae3d21-c7d1-42b4-b06d-aeeda703ca35 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:14.153070 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae3d21-c7d1-42b4-b06d-aeeda703ca35 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:14.154364 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae3d21-c7d1-42b4-b06d-aeeda703ca35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:14.230336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae3d21-c7d1-42b4-b06d-aeeda703ca35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:14.231748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae3d21-c7d1-42b4-b06d-aeeda703ca35 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:14.334724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 594/1183] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:14.341180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deea6a5-182f-444c-86d1-44540f28aac5 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:14.356840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:14.441132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deea6a5-182f-444c-86d1-44540f28aac5 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:14.478676 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723a7e2-4955-4324-bc6b-39d697bae82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:14.479330 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723a7e2-4955-4324-bc6b-39d697bae82a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:14.479608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723a7e2-4955-4324-bc6b-39d697bae82a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:14.479873 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723a7e2-4955-4324-bc6b-39d697bae82a None admin] PATH_INFO: `/v3/projects/aaff7fc01ada46c6b220cdd7d02eeb7c` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:14.481070 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723a7e2-4955-4324-bc6b-39d697bae82a None admin] RBAC: Authorizing `identity:get_project(project_id=aaff7fc01ada46c6b220cdd7d02eeb7c)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:14.482446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0723a7e2-4955-4324-bc6b-39d697bae82a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:14.484173 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 590/1184] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:14 2022] GET /identity/v3/projects/aaff7fc01ada46c6b220cdd7d02eeb7c => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:14.490649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae3d21-c7d1-42b4-b06d-aeeda703ca35 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:14.516041 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:14.592492 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae3d21-c7d1-42b4-b06d-aeeda703ca35 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:14.627826 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f9180-ca36-4a5e-a9f7-1e0c7d269f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:14.628435 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f9180-ca36-4a5e-a9f7-1e0c7d269f10 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:14.628765 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f9180-ca36-4a5e-a9f7-1e0c7d269f10 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:14.629040 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f9180-ca36-4a5e-a9f7-1e0c7d269f10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:14.629760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f9180-ca36-4a5e-a9f7-1e0c7d269f10 None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=aaff7fc01ada46c6b220cdd7d02eeb7c, user.id=e9091031eb324df0aad3f051ce7fdee6) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:14.630056 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f9180-ca36-4a5e-a9f7-1e0c7d269f10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:14.631530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f9180-ca36-4a5e-a9f7-1e0c7d269f10 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:14.637763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 595/1185] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:14 2022] GET /identity/v3/role_assignments?user.id=e9091031eb324df0aad3f051ce7fdee6&scope.project.id=aaff7fc01ada46c6b220cdd7d02eeb7c&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:16.797385 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235a730-15b1-4edd-bf6e-63f459149b51 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:16.797936 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235a730-15b1-4edd-bf6e-63f459149b51 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:16.797936 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235a730-15b1-4edd-bf6e-63f459149b51 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:16.799211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 591/1186] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:16.804405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce9595-22ce-4107-baeb-188c9026ea3a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:16.804843 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce9595-22ce-4107-baeb-188c9026ea3a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:16.804940 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce9595-22ce-4107-baeb-188c9026ea3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:16.823882 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-06ce9595-22ce-4107-baeb-188c9026ea3a None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:16.825199 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-06ce9595-22ce-4107-baeb-188c9026ea3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:16.893637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 596/1187] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:17.165647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235a730-15b1-4edd-bf6e-63f459149b51 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:17.247623 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:17.267653 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235a730-15b1-4edd-bf6e-63f459149b51 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:17.303176 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6312d-6ad2-45ed-87b0-a601950fe26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:17.303787 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6312d-6ad2-45ed-87b0-a601950fe26d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:17.304062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6312d-6ad2-45ed-87b0-a601950fe26d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:17.304328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6312d-6ad2-45ed-87b0-a601950fe26d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:17.314055 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6312d-6ad2-45ed-87b0-a601950fe26d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:17.318684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-59f6312d-6ad2-45ed-87b0-a601950fe26d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:17.318684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:17.318684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:17.318684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:17.318684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:17.318684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:17.318684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:17.318684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:17.318684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:17.318684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:17.318684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:17.318684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:17.318684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:17.318684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:17.318684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:17.318684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:17.320421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:17.320421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:17.320421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:17.320421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:17.320421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:17.320421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:17.320421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:17.320421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:17.320421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:17.320421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:17.320421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:17.320421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:17.320421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:17.320421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:17.320421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:17.320421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:17.321661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:17.321661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:17.321661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:17.321661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:17.321661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:17.321661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:17.321661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:17.321661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:17.321661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:17.321661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:17.321661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:17.321661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:40:17.321661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 592/1188] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:17.326681 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce9595-22ce-4107-baeb-188c9026ea3a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:17.345821 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:17.427308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce9595-22ce-4107-baeb-188c9026ea3a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:17.461834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be3967-0c81-41f3-928a-cf92a22ee2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:17.462440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be3967-0c81-41f3-928a-cf92a22ee2c6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:17.462758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be3967-0c81-41f3-928a-cf92a22ee2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:17.463329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be3967-0c81-41f3-928a-cf92a22ee2c6 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:17.463997 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be3967-0c81-41f3-928a-cf92a22ee2c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:17.464300 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be3967-0c81-41f3-928a-cf92a22ee2c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:17.465675 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be3967-0c81-41f3-928a-cf92a22ee2c6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:17.473786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 597/1189] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:17.482203 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6312d-6ad2-45ed-87b0-a601950fe26d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:17.580843 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6312d-6ad2-45ed-87b0-a601950fe26d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:17.613956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c817625-37c8-4abe-afac-4340b3252e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:17.614576 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c817625-37c8-4abe-afac-4340b3252e54 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:17.614877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c817625-37c8-4abe-afac-4340b3252e54 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:17.615168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c817625-37c8-4abe-afac-4340b3252e54 None admin] PATH_INFO: `/v3/users/e9091031eb324df0aad3f051ce7fdee6` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:17.616284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c817625-37c8-4abe-afac-4340b3252e54 None admin] RBAC: Authorizing `identity:get_user(user_id=e9091031eb324df0aad3f051ce7fdee6)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:17.618286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c817625-37c8-4abe-afac-4340b3252e54 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:17.619904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 593/1190] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:17 2022] GET /identity/v3/users/e9091031eb324df0aad3f051ce7fdee6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:17.625898 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be3967-0c81-41f3-928a-cf92a22ee2c6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:17.706599 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:17.725383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be3967-0c81-41f3-928a-cf92a22ee2c6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:17.761057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c6465-39bd-4b2e-ad91-eff3c1351fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:17.761673 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c6465-39bd-4b2e-ad91-eff3c1351fbf None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:17.761948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c6465-39bd-4b2e-ad91-eff3c1351fbf None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:17.762217 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c6465-39bd-4b2e-ad91-eff3c1351fbf None admin] PATH_INFO: `/v3/projects/aaff7fc01ada46c6b220cdd7d02eeb7c` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:17.763362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c6465-39bd-4b2e-ad91-eff3c1351fbf None admin] RBAC: Authorizing `identity:get_project(project_id=aaff7fc01ada46c6b220cdd7d02eeb7c)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:17.764779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c6465-39bd-4b2e-ad91-eff3c1351fbf None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:17.766396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 598/1191] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:17 2022] GET /identity/v3/projects/aaff7fc01ada46c6b220cdd7d02eeb7c => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:17.772664 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c817625-37c8-4abe-afac-4340b3252e54 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:17.784522 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:17.874577 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c817625-37c8-4abe-afac-4340b3252e54 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:17.909835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d850ec4-a822-4a13-87a1-3d378e806697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:17.910482 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d850ec4-a822-4a13-87a1-3d378e806697 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:17.910750 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d850ec4-a822-4a13-87a1-3d378e806697 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:17.911048 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d850ec4-a822-4a13-87a1-3d378e806697 None admin] PATH_INFO: `/v3/projects/aaff7fc01ada46c6b220cdd7d02eeb7c/users/e9091031eb324df0aad3f051ce7fdee6/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:17.912966 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d850ec4-a822-4a13-87a1-3d378e806697 None admin] RBAC: Authorizing `identity:create_grant(project_id=aaff7fc01ada46c6b220cdd7d02eeb7c, user_id=e9091031eb324df0aad3f051ce7fdee6, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:17.914952 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d850ec4-a822-4a13-87a1-3d378e806697 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:18.004711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 594/1192] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:40:17 2022] PUT /identity/v3/projects/aaff7fc01ada46c6b220cdd7d02eeb7c/users/e9091031eb324df0aad3f051ce7fdee6/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:40:20.204836 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc9457-a1e8-4fab-b28a-c93942706694 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:20.205403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc9457-a1e8-4fab-b28a-c93942706694 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:20.205403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc9457-a1e8-4fab-b28a-c93942706694 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:20.206809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 599/1193] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:20.212895 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01369e-2183-4583-ba8b-daf0b9ff1595 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:20.213600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01369e-2183-4583-ba8b-daf0b9ff1595 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:20.213708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01369e-2183-4583-ba8b-daf0b9ff1595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:20.232047 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ff01369e-2183-4583-ba8b-daf0b9ff1595 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:20.233216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ff01369e-2183-4583-ba8b-daf0b9ff1595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:20.294109 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:20.299371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 595/1194] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:20.572419 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc9457-a1e8-4fab-b28a-c93942706694 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:20.678027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc9457-a1e8-4fab-b28a-c93942706694 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:20.714659 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c9ad8-c626-4f81-9f0b-03c8062d45b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:20.715323 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c9ad8-c626-4f81-9f0b-03c8062d45b8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:20.715597 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c9ad8-c626-4f81-9f0b-03c8062d45b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:20.715865 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5c9ad8-c626-4f81-9f0b-03c8062d45b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:20.725844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5c9ad8-c626-4f81-9f0b-03c8062d45b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:20.731211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-6a5c9ad8-c626-4f81-9f0b-03c8062d45b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:20.731211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:20.731211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:20.731211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:20.731211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:20.731211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:20.731211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:20.731211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:20.731211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:20.731211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:20.731211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:20.731211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:20.731211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:20.731211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:20.731211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:20.731211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:20.732309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:20.732309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:20.732309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:20.732309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:20.732309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:20.732309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:20.732309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:20.732309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:20.732309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:20.732309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:20.732309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:20.732309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:20.732309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:20.732309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:20.732309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:20.732309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:20.733365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:20.733365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:20.733365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:20.733365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:20.733365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:20.733365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:20.733365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:20.733365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:20.733365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:20.733365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:20.733365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:20.733365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:40:20.733365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 600/1195] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:20.738611 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01369e-2183-4583-ba8b-daf0b9ff1595 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:20.839919 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01369e-2183-4583-ba8b-daf0b9ff1595 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:20.873991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014497a-4aa2-49f4-9b59-e7fb78c19a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:20.874594 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014497a-4aa2-49f4-9b59-e7fb78c19a19 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:20.874868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014497a-4aa2-49f4-9b59-e7fb78c19a19 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:20.875195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014497a-4aa2-49f4-9b59-e7fb78c19a19 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:20.875836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014497a-4aa2-49f4-9b59-e7fb78c19a19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:20.876131 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014497a-4aa2-49f4-9b59-e7fb78c19a19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:20.877497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014497a-4aa2-49f4-9b59-e7fb78c19a19 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:20.885881 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 596/1196] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:20.891907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c9ad8-c626-4f81-9f0b-03c8062d45b8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:20.903602 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:20.993400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5c9ad8-c626-4f81-9f0b-03c8062d45b8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:21.039504 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbe95a-78da-478a-b202-e07c2a0a2fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:21.040148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbe95a-78da-478a-b202-e07c2a0a2fb6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:21.040425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbe95a-78da-478a-b202-e07c2a0a2fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:21.040696 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbe95a-78da-478a-b202-e07c2a0a2fb6 None admin] PATH_INFO: `/v3/users/e9091031eb324df0aad3f051ce7fdee6` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:21.041823 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbe95a-78da-478a-b202-e07c2a0a2fb6 None admin] RBAC: Authorizing `identity:get_user(user_id=e9091031eb324df0aad3f051ce7fdee6)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:21.043289 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbe95a-78da-478a-b202-e07c2a0a2fb6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:21.044877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 601/1197] 10.222.0.91 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:20 2022] GET /identity/v3/users/e9091031eb324df0aad3f051ce7fdee6 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:21.051015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014497a-4aa2-49f4-9b59-e7fb78c19a19 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:21.125634 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:21.167674 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014497a-4aa2-49f4-9b59-e7fb78c19a19 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:21.204555 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8df434-bef5-4a40-a7a4-0402e26f4ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:21.205153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8df434-bef5-4a40-a7a4-0402e26f4ead None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:21.205426 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8df434-bef5-4a40-a7a4-0402e26f4ead None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:21.205691 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8df434-bef5-4a40-a7a4-0402e26f4ead None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:21.279952 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8df434-bef5-4a40-a7a4-0402e26f4ead None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:21.281309 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8df434-bef5-4a40-a7a4-0402e26f4ead None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:21.339928 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:21.383066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 597/1198] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:21.387974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbe95a-78da-478a-b202-e07c2a0a2fb6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:21.465458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:21.487861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbe95a-78da-478a-b202-e07c2a0a2fb6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:21.521951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1e847-9c7d-4875-8427-345d2816a7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:21.522574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1e847-9c7d-4875-8427-345d2816a7e8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:21.522853 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1e847-9c7d-4875-8427-345d2816a7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:21.523173 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1e847-9c7d-4875-8427-345d2816a7e8 None admin] PATH_INFO: `/v3/projects/aaff7fc01ada46c6b220cdd7d02eeb7c` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:21.524352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1e847-9c7d-4875-8427-345d2816a7e8 None admin] RBAC: Authorizing `identity:get_project(project_id=aaff7fc01ada46c6b220cdd7d02eeb7c)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:21.525894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1e847-9c7d-4875-8427-345d2816a7e8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:21.527572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 602/1199] 10.222.0.91 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:21 2022] GET /identity/v3/projects/aaff7fc01ada46c6b220cdd7d02eeb7c => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:21.534125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8df434-bef5-4a40-a7a4-0402e26f4ead None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:21.633219 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8df434-bef5-4a40-a7a4-0402e26f4ead None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:21.668936 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109d7c0-84ab-4ea0-a1a9-3278a20e25b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:21.669532 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109d7c0-84ab-4ea0-a1a9-3278a20e25b2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:21.669803 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109d7c0-84ab-4ea0-a1a9-3278a20e25b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:21.670068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109d7c0-84ab-4ea0-a1a9-3278a20e25b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:21.670776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109d7c0-84ab-4ea0-a1a9-3278a20e25b2 None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=aaff7fc01ada46c6b220cdd7d02eeb7c, user.id=e9091031eb324df0aad3f051ce7fdee6) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:21.671238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109d7c0-84ab-4ea0-a1a9-3278a20e25b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:21.672740 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109d7c0-84ab-4ea0-a1a9-3278a20e25b2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:21.681072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 598/1200] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:21 2022] GET /identity/v3/role_assignments?user.id=e9091031eb324df0aad3f051ce7fdee6&scope.project.id=aaff7fc01ada46c6b220cdd7d02eeb7c&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:31.536354 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=28 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:31.675971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=88 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:54.138705 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa454ab3-144d-40f5-8e2d-1bd7a1ad6435 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:54.139322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa454ab3-144d-40f5-8e2d-1bd7a1ad6435 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:54.139322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa454ab3-144d-40f5-8e2d-1bd7a1ad6435 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:54.140666 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 603/1201] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:41:54.146034 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e850663-3116-42f6-bfc4-6e41dd2a805c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:54.146496 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e850663-3116-42f6-bfc4-6e41dd2a805c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:54.146617 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e850663-3116-42f6-bfc4-6e41dd2a805c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:54.164886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-6e850663-3116-42f6-bfc4-6e41dd2a805c None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:41:54.166078 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6e850663-3116-42f6-bfc4-6e41dd2a805c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:41:54.233813 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 599/1202] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:41:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:41:54.506172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa454ab3-144d-40f5-8e2d-1bd7a1ad6435 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:54.611946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa454ab3-144d-40f5-8e2d-1bd7a1ad6435 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:54.648907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf6934-1917-4058-8a9f-ae5a6ffd5831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:54.649527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf6934-1917-4058-8a9f-ae5a6ffd5831 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:54.649803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf6934-1917-4058-8a9f-ae5a6ffd5831 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:54.650074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf6934-1917-4058-8a9f-ae5a6ffd5831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:54.660884 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf6934-1917-4058-8a9f-ae5a6ffd5831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:54.662361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf6934-1917-4058-8a9f-ae5a6ffd5831 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:54.681166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-6edf6934-1917-4058-8a9f-ae5a6ffd5831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:41:54.681166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:41:54.681166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:41:54.681166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:41:54.681166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:41:54.681166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:41:54.681166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:41:54.681166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:41:54.681166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:41:54.681166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:41:54.681166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:41:54.681166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:41:54.681166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:41:54.681166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:41:54.681166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:41:54.681166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:41:54.682460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:41:54.683488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:41:54.683488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:41:54.683488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:41:54.683488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 604/1203] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:41:54.688569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e850663-3116-42f6-bfc4-6e41dd2a805c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:54.789466 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e850663-3116-42f6-bfc4-6e41dd2a805c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:54.817982 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:54.823937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d897397-b079-470f-9025-4415bc6c947a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:54.824563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d897397-b079-470f-9025-4415bc6c947a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:54.824934 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d897397-b079-470f-9025-4415bc6c947a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:54.825243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d897397-b079-470f-9025-4415bc6c947a None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:54.825902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d897397-b079-470f-9025-4415bc6c947a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:54.826205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d897397-b079-470f-9025-4415bc6c947a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:54.828095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d897397-b079-470f-9025-4415bc6c947a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:54.839256 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 600/1204] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:54.845785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf6934-1917-4058-8a9f-ae5a6ffd5831 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:54.898462 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:54.945741 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf6934-1917-4058-8a9f-ae5a6ffd5831 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:54.981001 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6b3ef-0f7e-4e4c-b489-e45378548dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:54.981697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6b3ef-0f7e-4e4c-b489-e45378548dcd None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:54.981943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6b3ef-0f7e-4e4c-b489-e45378548dcd None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:54.982219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6b3ef-0f7e-4e4c-b489-e45378548dcd None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:54.982882 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6b3ef-0f7e-4e4c-b489-e45378548dcd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:54.984359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6b3ef-0f7e-4e4c-b489-e45378548dcd None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:55.338483 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 605/1205] 10.222.0.91 () {60 vars in 1132 bytes} [Thu Sep 8 13:41:54 2022] POST /identity/v3/users => generated 251 bytes in 493 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:41:57.583215 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90b692-1351-4e2f-b2cf-93e302fbab4b None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:57.584492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90b692-1351-4e2f-b2cf-93e302fbab4b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:57.584492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90b692-1351-4e2f-b2cf-93e302fbab4b None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:57.585735 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 601/1206] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:41:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:41:57.591025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc7d1d-1866-4b8a-9b4e-5a7d08ae04cd None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:57.591474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc7d1d-1866-4b8a-9b4e-5a7d08ae04cd None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:57.591598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc7d1d-1866-4b8a-9b4e-5a7d08ae04cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:57.611104 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-89dc7d1d-1866-4b8a-9b4e-5a7d08ae04cd None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:41:57.612287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-89dc7d1d-1866-4b8a-9b4e-5a7d08ae04cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:41:57.682085 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 606/1207] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:41:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:41:57.959746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90b692-1351-4e2f-b2cf-93e302fbab4b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:58.069634 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90b692-1351-4e2f-b2cf-93e302fbab4b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:58.102698 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7db87-6174-4f55-9209-6fbc40c2f2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:58.105816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7db87-6174-4f55-9209-6fbc40c2f2e7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:58.105816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7db87-6174-4f55-9209-6fbc40c2f2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:58.105816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7db87-6174-4f55-9209-6fbc40c2f2e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:58.114741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c7db87-6174-4f55-9209-6fbc40c2f2e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:41:58.119411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-09c7db87-6174-4f55-9209-6fbc40c2f2e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:41:58.119411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:41:58.119411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:41:58.119411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:41:58.119411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:41:58.119411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:41:58.119411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:41:58.119411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:41:58.119411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:41:58.119411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:41:58.119411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:41:58.119411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:41:58.119411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:41:58.119411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:41:58.119411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:41:58.119411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:41:58.120503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:41:58.120503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:41:58.120503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:41:58.120503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:41:58.120503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:41:58.120503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:41:58.120503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:41:58.120503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:41:58.120503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:41:58.120503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:41:58.120503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:41:58.120503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:41:58.120503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:41:58.120503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:41:58.120503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:41:58.120503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:41:58.121368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:41:58.121368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:41:58.121368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:41:58.121368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:41:58.121368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:41:58.121368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:41:58.121368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:41:58.121368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:41:58.121368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:41:58.121368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:41:58.121368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:41:58.121368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:41:58.121368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 602/1208] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:41:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:41:58.126652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc7d1d-1866-4b8a-9b4e-5a7d08ae04cd None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:58.165993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:58.226536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc7d1d-1866-4b8a-9b4e-5a7d08ae04cd None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:58.260116 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e17530-2df5-4dbd-8d98-39ebfc44f926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:58.260718 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e17530-2df5-4dbd-8d98-39ebfc44f926 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:58.261041 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e17530-2df5-4dbd-8d98-39ebfc44f926 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:58.261355 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e17530-2df5-4dbd-8d98-39ebfc44f926 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:58.262092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e17530-2df5-4dbd-8d98-39ebfc44f926 None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:58.262395 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e17530-2df5-4dbd-8d98-39ebfc44f926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:58.263793 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e17530-2df5-4dbd-8d98-39ebfc44f926 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:58.272531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 607/1209] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:41:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:58.280355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7db87-6174-4f55-9209-6fbc40c2f2e7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:58.338819 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:58.388077 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7db87-6174-4f55-9209-6fbc40c2f2e7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:58.424569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878a888-de60-4348-8f9f-bd9a2f015752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:58.425180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878a888-de60-4348-8f9f-bd9a2f015752 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:58.425452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878a888-de60-4348-8f9f-bd9a2f015752 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:58.425720 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878a888-de60-4348-8f9f-bd9a2f015752 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:58.436445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878a888-de60-4348-8f9f-bd9a2f015752 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:58.438440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878a888-de60-4348-8f9f-bd9a2f015752 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:58.452933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-1878a888-de60-4348-8f9f-bd9a2f015752 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:41:58.452933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:41:58.452933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:41:58.452933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:41:58.452933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:41:58.452933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:41:58.452933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:41:58.452933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:41:58.452933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:41:58.452933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:41:58.452933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:41:58.452933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:41:58.452933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:41:58.452933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:41:58.452933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:41:58.452933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:41:58.453994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:41:58.454880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:41:58.454880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:41:58.454880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:41:58.454880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 603/1210] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:41:58.459856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e17530-2df5-4dbd-8d98-39ebfc44f926 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:58.569367 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:58.574143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e17530-2df5-4dbd-8d98-39ebfc44f926 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:58.611812 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decddcc-1f12-4b36-9e0d-2ac872bf7c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:58.612518 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decddcc-1f12-4b36-9e0d-2ac872bf7c0b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:58.612689 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decddcc-1f12-4b36-9e0d-2ac872bf7c0b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:58.612973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decddcc-1f12-4b36-9e0d-2ac872bf7c0b None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:58.613617 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decddcc-1f12-4b36-9e0d-2ac872bf7c0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:58.613918 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decddcc-1f12-4b36-9e0d-2ac872bf7c0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:58.615336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decddcc-1f12-4b36-9e0d-2ac872bf7c0b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:58.629412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 608/1211] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:58.635305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878a888-de60-4348-8f9f-bd9a2f015752 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:58.739984 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878a888-de60-4348-8f9f-bd9a2f015752 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:58.741628 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:58.774444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad24f2-d1b7-430d-947e-aff5177b27aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:58.775105 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad24f2-d1b7-430d-947e-aff5177b27aa None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:58.775380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad24f2-d1b7-430d-947e-aff5177b27aa None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:58.775651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad24f2-d1b7-430d-947e-aff5177b27aa None admin] PATH_INFO: `/v3/users/placement` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:58.783647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad24f2-d1b7-430d-947e-aff5177b27aa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:58.792027 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad24f2-d1b7-430d-947e-aff5177b27aa None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:58.796088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-43ad24f2-d1b7-430d-947e-aff5177b27aa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:41:58.796088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:41:58.796088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:41:58.796088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:41:58.796088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:41:58.796088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:41:58.796088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:41:58.796088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:41:58.796088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:41:58.796088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:41:58.796088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:41:58.796088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:41:58.796088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:41:58.796088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:41:58.796088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:41:58.796088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:41:58.797745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:41:58.798851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:41:58.798851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:41:58.798851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:41:58.798851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:41:58.798851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:41:58.798851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:41:58.798851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:41:58.798851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:41:58.798851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:41:58.798851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:41:58.798851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 604/1212] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:41:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:41:58.803702 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decddcc-1f12-4b36-9e0d-2ac872bf7c0b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:58.915874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decddcc-1f12-4b36-9e0d-2ac872bf7c0b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:58.958111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7271eb-1cba-4897-9039-c4c3f6c29177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:58.959222 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7271eb-1cba-4897-9039-c4c3f6c29177 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:58.959630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7271eb-1cba-4897-9039-c4c3f6c29177 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:58.959909 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7271eb-1cba-4897-9039-c4c3f6c29177 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:58.960637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7271eb-1cba-4897-9039-c4c3f6c29177 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:58.960943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7271eb-1cba-4897-9039-c4c3f6c29177 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:58.962371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7271eb-1cba-4897-9039-c4c3f6c29177 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:58.981342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 609/1213] 10.222.0.91 () {56 vars in 1143 bytes} [Thu Sep 8 13:41:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:58.988116 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad24f2-d1b7-430d-947e-aff5177b27aa None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:59.105795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad24f2-d1b7-430d-947e-aff5177b27aa None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:59.147076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbbf8d-fab4-4658-987a-f55620e755bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:59.147664 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbbf8d-fab4-4658-987a-f55620e755bc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:59.147960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbbf8d-fab4-4658-987a-f55620e755bc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:59.148210 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbbf8d-fab4-4658-987a-f55620e755bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:59.214881 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbbf8d-fab4-4658-987a-f55620e755bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:59.216354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbbf8d-fab4-4658-987a-f55620e755bc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:59.312628 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 605/1214] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:41:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:41:59.314939 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:59.319503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7271eb-1cba-4897-9039-c4c3f6c29177 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:59.338627 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:59.422601 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7271eb-1cba-4897-9039-c4c3f6c29177 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:59.456060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3377e5-c18f-4555-b568-baca4dc34a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:59.456679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3377e5-c18f-4555-b568-baca4dc34a40 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:59.456949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3377e5-c18f-4555-b568-baca4dc34a40 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:59.457215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3377e5-c18f-4555-b568-baca4dc34a40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:59.467261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3377e5-c18f-4555-b568-baca4dc34a40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:59.469298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3377e5-c18f-4555-b568-baca4dc34a40 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:59.489590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-5a3377e5-c18f-4555-b568-baca4dc34a40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:41:59.489590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:41:59.489590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:41:59.489590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:41:59.489590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:41:59.489590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:41:59.489590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:41:59.489590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:41:59.489590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:41:59.489590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:41:59.489590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:41:59.489590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:41:59.489590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:41:59.489590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:41:59.489590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:41:59.489590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:41:59.490735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:41:59.491707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:41:59.491707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:41:59.491707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:41:59.491707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 610/1215] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:41:59.497792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbbf8d-fab4-4658-987a-f55620e755bc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:59.567729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:59.601041 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbbf8d-fab4-4658-987a-f55620e755bc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:59.637253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ffa19-ac9c-4105-88db-6f510d66f7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:59.637972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ffa19-ac9c-4105-88db-6f510d66f7f7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:59.638118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ffa19-ac9c-4105-88db-6f510d66f7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:59.638387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ffa19-ac9c-4105-88db-6f510d66f7f7 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:59.639058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ffa19-ac9c-4105-88db-6f510d66f7f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:59.639359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ffa19-ac9c-4105-88db-6f510d66f7f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:59.641401 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ffa19-ac9c-4105-88db-6f510d66f7f7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:59.653295 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 606/1216] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:41:59.659329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3377e5-c18f-4555-b568-baca4dc34a40 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:59.770469 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3377e5-c18f-4555-b568-baca4dc34a40 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:59.803837 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc6afd-67ba-46ab-8942-ab4ad50eac28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:59.804578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc6afd-67ba-46ab-8942-ab4ad50eac28 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:59.804959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc6afd-67ba-46ab-8942-ab4ad50eac28 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:59.805209 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc6afd-67ba-46ab-8942-ab4ad50eac28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:59.812887 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc6afd-67ba-46ab-8942-ab4ad50eac28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:59.814390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc6afd-67ba-46ab-8942-ab4ad50eac28 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:59.825994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-3ccc6afd-67ba-46ab-8942-ab4ad50eac28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:41:59.825994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:41:59.825994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:41:59.825994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:41:59.825994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:41:59.825994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:41:59.825994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:41:59.825994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:41:59.825994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:41:59.825994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:41:59.825994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:41:59.825994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:41:59.825994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:41:59.825994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:41:59.825994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:41:59.827063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:41:59.828905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:41:59.828905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:41:59.828905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:41:59.828905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:41:59.828905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:41:59.828905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:41:59.828905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:41:59.828905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:41:59.828905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 611/1217] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:41:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:41:59.833859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ffa19-ac9c-4105-88db-6f510d66f7f7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:59.834747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:59.941972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ffa19-ac9c-4105-88db-6f510d66f7f7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:59.976383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639eb42-7282-45a0-992d-4082206dee28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:59.976976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639eb42-7282-45a0-992d-4082206dee28 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:59.977252 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639eb42-7282-45a0-992d-4082206dee28 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:59.977523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639eb42-7282-45a0-992d-4082206dee28 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:59.978210 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639eb42-7282-45a0-992d-4082206dee28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:59.978516 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639eb42-7282-45a0-992d-4082206dee28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:59.979977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639eb42-7282-45a0-992d-4082206dee28 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:59.992370 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 607/1218] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:41:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:41:59.998590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc6afd-67ba-46ab-8942-ab4ad50eac28 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:00.104628 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:00.107386 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc6afd-67ba-46ab-8942-ab4ad50eac28 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:00.142428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df044a5-7e62-4608-a35b-755062fca4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:00.143131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df044a5-7e62-4608-a35b-755062fca4f3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:00.143406 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df044a5-7e62-4608-a35b-755062fca4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:00.143674 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df044a5-7e62-4608-a35b-755062fca4f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:00.144386 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df044a5-7e62-4608-a35b-755062fca4f3 None admin] RBAC: Adding query filter params (role.id=44e187abae1d4b248fd145ad9cb97ed2, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=f9506139d5744fe69153e7e30a05b436) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:00.144688 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df044a5-7e62-4608-a35b-755062fca4f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:00.146138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df044a5-7e62-4608-a35b-755062fca4f3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:00.151936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 612/1219] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:41:59 2022] GET /identity/v3/role_assignments?user.id=f9506139d5744fe69153e7e30a05b436&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=44e187abae1d4b248fd145ad9cb97ed2 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:00.252714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:02.312584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1aa4d9-5b4f-4146-b631-23300a249606 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:02.313158 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1aa4d9-5b4f-4146-b631-23300a249606 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:02.313158 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1aa4d9-5b4f-4146-b631-23300a249606 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:02.314494 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 608/1220] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:02.320006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173fb11-6243-45a2-9fbf-88da5ef7a5c8 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:02.320523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173fb11-6243-45a2-9fbf-88da5ef7a5c8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:02.320523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173fb11-6243-45a2-9fbf-88da5ef7a5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:02.339661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d173fb11-6243-45a2-9fbf-88da5ef7a5c8 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:02.340849 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d173fb11-6243-45a2-9fbf-88da5ef7a5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:02.414114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 613/1221] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:02.684886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1aa4d9-5b4f-4146-b631-23300a249606 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:02.792105 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1aa4d9-5b4f-4146-b631-23300a249606 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:02.828994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6cfd3-68b8-484f-88a5-df8552a8a379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:02.829613 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6cfd3-68b8-484f-88a5-df8552a8a379 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:02.829888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6cfd3-68b8-484f-88a5-df8552a8a379 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:02.830162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6cfd3-68b8-484f-88a5-df8552a8a379 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:02.841090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6cfd3-68b8-484f-88a5-df8552a8a379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:02.846199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-fdc6cfd3-68b8-484f-88a5-df8552a8a379 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:42:02.846199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:02.846199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:02.846199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:02.846199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:02.846199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:02.846199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:02.846199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:02.846199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:02.846199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:02.846199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:02.846199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:02.846199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:02.846199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:02.846199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:02.846199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:02.847684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:02.847684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:02.847684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:02.847684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:02.847684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:02.847684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:02.847684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:02.847684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:02.847684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:02.847684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:02.847684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:02.847684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:02.847684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:02.847684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:02.847684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:02.847684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:02.848544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:02.848544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:02.848544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:02.848544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:02.848544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:02.848544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:02.848544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:02.848544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:02.848544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:02.848544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:02.848544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:42:02.848544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:02.848544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 609/1222] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:42:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:02.853779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173fb11-6243-45a2-9fbf-88da5ef7a5c8 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:02.953957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d173fb11-6243-45a2-9fbf-88da5ef7a5c8 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:02.977419 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:02.988241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081f1bc-7bbc-42e1-a58d-ebbca4d82bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:02.988921 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081f1bc-7bbc-42e1-a58d-ebbca4d82bef None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:02.989198 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081f1bc-7bbc-42e1-a58d-ebbca4d82bef None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:02.989473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081f1bc-7bbc-42e1-a58d-ebbca4d82bef None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:02.990149 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081f1bc-7bbc-42e1-a58d-ebbca4d82bef None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:02.990432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081f1bc-7bbc-42e1-a58d-ebbca4d82bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:02.991831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081f1bc-7bbc-42e1-a58d-ebbca4d82bef None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:03.000534 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 614/1223] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:42:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:03.008343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6cfd3-68b8-484f-88a5-df8552a8a379 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:03.055514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:03.113739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6cfd3-68b8-484f-88a5-df8552a8a379 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:03.148568 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509ed7c-1057-4ebd-8499-65d954f82c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:03.149210 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509ed7c-1057-4ebd-8499-65d954f82c94 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:03.149487 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509ed7c-1057-4ebd-8499-65d954f82c94 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:03.149756 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509ed7c-1057-4ebd-8499-65d954f82c94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:03.159587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509ed7c-1057-4ebd-8499-65d954f82c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:03.161024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509ed7c-1057-4ebd-8499-65d954f82c94 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:03.175917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-9509ed7c-1057-4ebd-8499-65d954f82c94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:03.175917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:03.175917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:03.175917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:03.175917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:03.175917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:03.175917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:03.175917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:03.175917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:03.175917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:03.175917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:03.175917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:03.175917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:03.175917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:03.175917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:03.175917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:03.177129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:03.178087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:03.178087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:03.178087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:03.178087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 610/1224] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:03.183117 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081f1bc-7bbc-42e1-a58d-ebbca4d82bef None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:03.282605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081f1bc-7bbc-42e1-a58d-ebbca4d82bef None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:03.319963 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5889f-48a1-405b-bb5e-ecf8a27a410b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:03.320620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5889f-48a1-405b-bb5e-ecf8a27a410b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:03.321176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5889f-48a1-405b-bb5e-ecf8a27a410b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:03.321176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5889f-48a1-405b-bb5e-ecf8a27a410b None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:03.321783 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5889f-48a1-405b-bb5e-ecf8a27a410b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:03.322086 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5889f-48a1-405b-bb5e-ecf8a27a410b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:03.323493 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5889f-48a1-405b-bb5e-ecf8a27a410b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:03.335473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 615/1225] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:03.341230 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509ed7c-1057-4ebd-8499-65d954f82c94 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:03.437880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:03.444046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509ed7c-1057-4ebd-8499-65d954f82c94 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:03.477643 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc6696-8f0d-40d7-a5f5-9131068ffa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:03.478387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc6696-8f0d-40d7-a5f5-9131068ffa4b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:03.479077 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc6696-8f0d-40d7-a5f5-9131068ffa4b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:03.479237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc6696-8f0d-40d7-a5f5-9131068ffa4b None admin] PATH_INFO: `/v3/users/placement` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:03.488136 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc6696-8f0d-40d7-a5f5-9131068ffa4b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:03.489708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc6696-8f0d-40d7-a5f5-9131068ffa4b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:03.500977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-eedc6696-8f0d-40d7-a5f5-9131068ffa4b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:03.500977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:03.500977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:03.500977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:03.500977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:03.500977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:03.500977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:03.500977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:03.500977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:03.500977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:03.500977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:03.500977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:03.500977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:03.500977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:03.500977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:03.500977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:03.502128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:03.503201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:03.503201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:03.503201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:03.503201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:03.503201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:03.503201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:03.503201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:03.503201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:03.503201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:03.503201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:03.503201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 611/1226] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:03.508144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5889f-48a1-405b-bb5e-ecf8a27a410b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:03.559408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:03.616287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5889f-48a1-405b-bb5e-ecf8a27a410b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:03.650784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058853c-e1b5-4a08-8069-8ca6660d1976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:03.651412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058853c-e1b5-4a08-8069-8ca6660d1976 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:03.651685 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058853c-e1b5-4a08-8069-8ca6660d1976 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:03.651953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058853c-e1b5-4a08-8069-8ca6660d1976 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:03.652669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058853c-e1b5-4a08-8069-8ca6660d1976 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:03.652969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058853c-e1b5-4a08-8069-8ca6660d1976 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:03.654401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058853c-e1b5-4a08-8069-8ca6660d1976 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:03.672493 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 616/1227] 10.222.0.91 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:03.678350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc6696-8f0d-40d7-a5f5-9131068ffa4b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:03.781883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc6696-8f0d-40d7-a5f5-9131068ffa4b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:03.824700 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59823c2-28e5-444c-8a33-c22050bef347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:03.825301 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59823c2-28e5-444c-8a33-c22050bef347 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:03.825571 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59823c2-28e5-444c-8a33-c22050bef347 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:03.825841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59823c2-28e5-444c-8a33-c22050bef347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:03.837560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59823c2-28e5-444c-8a33-c22050bef347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:03.839178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59823c2-28e5-444c-8a33-c22050bef347 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:03.854004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-b59823c2-28e5-444c-8a33-c22050bef347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:03.854004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:03.854004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:03.854004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:03.854004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:03.854004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:03.854004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:03.854004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:03.854004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:03.854004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:03.854004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:03.854004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:03.854004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:03.854004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:03.854004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:03.854004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:03.855304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:03.856359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:03.856359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:03.856359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:03.856359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 612/1228] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:03.861016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058853c-e1b5-4a08-8069-8ca6660d1976 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:03.976846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058853c-e1b5-4a08-8069-8ca6660d1976 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:03.979174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:04.015197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d47933-bccf-4a5f-83ad-af30e6ea0453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:04.016338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d47933-bccf-4a5f-83ad-af30e6ea0453 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:04.016612 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d47933-bccf-4a5f-83ad-af30e6ea0453 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:04.016881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d47933-bccf-4a5f-83ad-af30e6ea0453 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:04.017512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d47933-bccf-4a5f-83ad-af30e6ea0453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:04.017812 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d47933-bccf-4a5f-83ad-af30e6ea0453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:04.019198 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d47933-bccf-4a5f-83ad-af30e6ea0453 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:04.033500 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 617/1229] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:04.039255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59823c2-28e5-444c-8a33-c22050bef347 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:04.049144 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:04.145487 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59823c2-28e5-444c-8a33-c22050bef347 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:04.180205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b504e3b-ac97-4185-890e-7787f939f581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:04.180831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b504e3b-ac97-4185-890e-7787f939f581 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:04.181103 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b504e3b-ac97-4185-890e-7787f939f581 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:04.181370 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b504e3b-ac97-4185-890e-7787f939f581 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:04.188629 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b504e3b-ac97-4185-890e-7787f939f581 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:04.190069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b504e3b-ac97-4185-890e-7787f939f581 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:04.201165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-3b504e3b-ac97-4185-890e-7787f939f581 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:04.201165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:04.201165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:04.201165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:04.201165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:04.201165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:04.201165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:04.201165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:04.201165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:04.201165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:04.201165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:04.201165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:04.201165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:42:04.201165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:42:04.201165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:04.202269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:04.203800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:42:04.203800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:42:04.203800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:42:04.203800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:42:04.203800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:42:04.203800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:42:04.203800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:04.203800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:04.203800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 613/1230] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:42:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:04.208779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d47933-bccf-4a5f-83ad-af30e6ea0453 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:04.319353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d47933-bccf-4a5f-83ad-af30e6ea0453 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:04.354156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae14295-ba3c-40b4-94f0-dc1a8fb2e665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:04.354828 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae14295-ba3c-40b4-94f0-dc1a8fb2e665 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:04.355075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae14295-ba3c-40b4-94f0-dc1a8fb2e665 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:04.355353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae14295-ba3c-40b4-94f0-dc1a8fb2e665 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:04.356045 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae14295-ba3c-40b4-94f0-dc1a8fb2e665 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:04.356353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae14295-ba3c-40b4-94f0-dc1a8fb2e665 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:04.357793 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae14295-ba3c-40b4-94f0-dc1a8fb2e665 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:04.369935 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 618/1231] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:42:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:04.376600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b504e3b-ac97-4185-890e-7787f939f581 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:04.443640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:04.478722 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b504e3b-ac97-4185-890e-7787f939f581 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:04.512924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae24da-44e1-4606-adb4-bdd9a633dc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:04.513559 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae24da-44e1-4606-adb4-bdd9a633dc0e None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:04.513831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae24da-44e1-4606-adb4-bdd9a633dc0e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:04.514100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae24da-44e1-4606-adb4-bdd9a633dc0e None admin] PATH_INFO: `/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/f9506139d5744fe69153e7e30a05b436/roles/44e187abae1d4b248fd145ad9cb97ed2` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:04.528731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae24da-44e1-4606-adb4-bdd9a633dc0e None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb2bd53d40841eeb1653bd5927fb189, user_id=f9506139d5744fe69153e7e30a05b436, role_id=44e187abae1d4b248fd145ad9cb97ed2)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:04.530158 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae24da-44e1-4606-adb4-bdd9a633dc0e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:04.732967 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 614/1232] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:42:04 2022] PUT /identity/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/f9506139d5744fe69153e7e30a05b436/roles/44e187abae1d4b248fd145ad9cb97ed2 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:42:06.917645 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71339b9-0ca4-4834-9d46-2e86133458e6 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:06.918198 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71339b9-0ca4-4834-9d46-2e86133458e6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:06.918198 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71339b9-0ca4-4834-9d46-2e86133458e6 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:06.919572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 619/1233] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:06.925757 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d278a9-78f3-44a0-a2f3-eb20ba65c266 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:06.931452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d278a9-78f3-44a0-a2f3-eb20ba65c266 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:06.931452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d278a9-78f3-44a0-a2f3-eb20ba65c266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:06.945253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-18d278a9-78f3-44a0-a2f3-eb20ba65c266 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:06.946439 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-18d278a9-78f3-44a0-a2f3-eb20ba65c266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:07.018674 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 615/1234] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:07.293134 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71339b9-0ca4-4834-9d46-2e86133458e6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:07.328034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:07.412649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71339b9-0ca4-4834-9d46-2e86133458e6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:07.449902 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccef63-9d99-4acb-b5c8-fd33799eada8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:07.450536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccef63-9d99-4acb-b5c8-fd33799eada8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:07.450821 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccef63-9d99-4acb-b5c8-fd33799eada8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:07.451200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccef63-9d99-4acb-b5c8-fd33799eada8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:07.462914 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccef63-9d99-4acb-b5c8-fd33799eada8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:07.467800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-12ccef63-9d99-4acb-b5c8-fd33799eada8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:42:07.467800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:07.467800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:07.467800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:07.467800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:07.467800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:07.467800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:07.467800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:07.467800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:07.467800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:07.467800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:07.467800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:07.467800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:07.467800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:07.467800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:07.467800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:07.470369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:07.470369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:07.470369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:07.470369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:07.470369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:07.470369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:07.470369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:07.470369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:07.470369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:07.470369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:07.470369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:07.470369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:07.470369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:07.470369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:07.470369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:07.470369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:07.472495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:07.472495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:07.472495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:07.472495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:07.472495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:07.472495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:07.472495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:07.472495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:07.472495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:07.472495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:07.472495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:42:07.472495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:42:07.472495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 620/1235] 10.222.0.91 () {56 vars in 1102 bytes} [Thu Sep 8 13:42:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:07.476547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d278a9-78f3-44a0-a2f3-eb20ba65c266 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:07.536934 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:07.591305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d278a9-78f3-44a0-a2f3-eb20ba65c266 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:07.628952 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633eb5e-21fd-4b86-9798-d47cf2fc682e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:07.629553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633eb5e-21fd-4b86-9798-d47cf2fc682e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:07.629826 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633eb5e-21fd-4b86-9798-d47cf2fc682e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:07.630091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633eb5e-21fd-4b86-9798-d47cf2fc682e None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:07.630795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633eb5e-21fd-4b86-9798-d47cf2fc682e None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:07.631202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633eb5e-21fd-4b86-9798-d47cf2fc682e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:07.632579 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633eb5e-21fd-4b86-9798-d47cf2fc682e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:07.641514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 616/1236] 10.222.0.91 () {56 vars in 1103 bytes} [Thu Sep 8 13:42:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:07.648657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccef63-9d99-4acb-b5c8-fd33799eada8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:07.745377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:07.762258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccef63-9d99-4acb-b5c8-fd33799eada8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:07.800340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22ba01-c5cd-4a58-8e00-ebd0f9fdcf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:07.800970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22ba01-c5cd-4a58-8e00-ebd0f9fdcf18 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:07.801253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22ba01-c5cd-4a58-8e00-ebd0f9fdcf18 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:07.801520 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22ba01-c5cd-4a58-8e00-ebd0f9fdcf18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:07.812221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22ba01-c5cd-4a58-8e00-ebd0f9fdcf18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:07.814183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22ba01-c5cd-4a58-8e00-ebd0f9fdcf18 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:07.831554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-1e22ba01-c5cd-4a58-8e00-ebd0f9fdcf18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:07.831554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:07.831554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:07.831554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:07.831554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:07.831554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:07.831554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:07.831554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:07.831554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:07.831554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:07.831554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:07.831554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:07.831554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:07.831554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:07.831554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:07.831554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:07.832804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:07.833955 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:07.833955 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:07.833955 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:42:07.833955 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 621/1237] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:07.838814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633eb5e-21fd-4b86-9798-d47cf2fc682e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:07.942786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633eb5e-21fd-4b86-9798-d47cf2fc682e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:07.953187 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:07.980373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3041-0df0-4ae5-8131-534e731facb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:07.980975 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3041-0df0-4ae5-8131-534e731facb0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:07.981245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3041-0df0-4ae5-8131-534e731facb0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:07.981514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b3041-0df0-4ae5-8131-534e731facb0 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:07.982148 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b3041-0df0-4ae5-8131-534e731facb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:07.982445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b3041-0df0-4ae5-8131-534e731facb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:07.983851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b3041-0df0-4ae5-8131-534e731facb0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:07.996475 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 617/1238] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:08.003309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22ba01-c5cd-4a58-8e00-ebd0f9fdcf18 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:08.107743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22ba01-c5cd-4a58-8e00-ebd0f9fdcf18 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:08.152740 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64a3dd-26b3-4f7d-8113-bc0b04540bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:08.153381 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64a3dd-26b3-4f7d-8113-bc0b04540bd5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:08.153657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64a3dd-26b3-4f7d-8113-bc0b04540bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:08.153931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64a3dd-26b3-4f7d-8113-bc0b04540bd5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:08.156791 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:08.161887 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64a3dd-26b3-4f7d-8113-bc0b04540bd5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:08.163383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64a3dd-26b3-4f7d-8113-bc0b04540bd5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:08.175528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-de64a3dd-26b3-4f7d-8113-bc0b04540bd5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:08.175528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:08.175528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:08.175528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:08.175528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:08.175528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:08.175528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:08.175528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:08.175528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:08.175528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:08.175528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:08.175528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:08.175528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:08.175528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:08.175528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:08.175528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:08.177088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:08.179264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:08.179264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:08.179264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:08.179264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:08.179264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:08.179264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:08.179264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:08.179264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:08.179264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:08.179264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:42:08.179264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 622/1239] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:08.184346 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3041-0df0-4ae5-8131-534e731facb0 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:08.303566 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b3041-0df0-4ae5-8131-534e731facb0 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:08.359732 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11815a6-32ec-4406-818a-74bde340daf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:08.360400 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11815a6-32ec-4406-818a-74bde340daf4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:08.360672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11815a6-32ec-4406-818a-74bde340daf4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:08.360935 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11815a6-32ec-4406-818a-74bde340daf4 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:08.361658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11815a6-32ec-4406-818a-74bde340daf4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:08.361985 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11815a6-32ec-4406-818a-74bde340daf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:08.363467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11815a6-32ec-4406-818a-74bde340daf4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:08.383774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 618/1240] 10.222.0.91 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:08.393266 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64a3dd-26b3-4f7d-8113-bc0b04540bd5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:08.533955 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64a3dd-26b3-4f7d-8113-bc0b04540bd5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:08.574527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728d345-aa99-4f89-bb19-67edca9b79bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:08.575136 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728d345-aa99-4f89-bb19-67edca9b79bd None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:08.575450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728d345-aa99-4f89-bb19-67edca9b79bd None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:08.575755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728d345-aa99-4f89-bb19-67edca9b79bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:08.655710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:08.663548 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728d345-aa99-4f89-bb19-67edca9b79bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:08.665017 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728d345-aa99-4f89-bb19-67edca9b79bd None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:08.796670 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 623/1241] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:42:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:42:08.803777 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11815a6-32ec-4406-818a-74bde340daf4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:08.836885 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:08.932448 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11815a6-32ec-4406-818a-74bde340daf4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:08.978089 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0b5cf-8cf3-4959-9661-41859d157cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:08.978960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0b5cf-8cf3-4959-9661-41859d157cf9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:08.979057 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0b5cf-8cf3-4959-9661-41859d157cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:08.979288 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0b5cf-8cf3-4959-9661-41859d157cf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:08.990336 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0b5cf-8cf3-4959-9661-41859d157cf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:08.991867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0b5cf-8cf3-4959-9661-41859d157cf9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:09.007745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-11c0b5cf-8cf3-4959-9661-41859d157cf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:09.007745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:09.007745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:09.007745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:09.007745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:09.007745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:09.007745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:09.007745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:09.007745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:09.007745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:09.007745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:09.007745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:09.007745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:09.007745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:09.007745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:09.007745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:09.010201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:09.011336 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:09.011336 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:09.011336 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:09.011336 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 619/1242] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:09.019344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728d345-aa99-4f89-bb19-67edca9b79bd None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:09.143802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:09.149943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728d345-aa99-4f89-bb19-67edca9b79bd None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:09.196115 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f0fe5-b74e-447f-b285-7516808bd8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:09.196730 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f0fe5-b74e-447f-b285-7516808bd8e5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:09.197013 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f0fe5-b74e-447f-b285-7516808bd8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:09.197287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f0fe5-b74e-447f-b285-7516808bd8e5 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:09.197938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f0fe5-b74e-447f-b285-7516808bd8e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:09.198248 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f0fe5-b74e-447f-b285-7516808bd8e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:09.199734 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f0fe5-b74e-447f-b285-7516808bd8e5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:09.213082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 624/1243] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:09.219859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0b5cf-8cf3-4959-9661-41859d157cf9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:09.316821 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:09.339312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0b5cf-8cf3-4959-9661-41859d157cf9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:09.394743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf320d-f3d3-4eb9-b0ce-0203b7d455d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:09.395495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf320d-f3d3-4eb9-b0ce-0203b7d455d0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:09.395781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf320d-f3d3-4eb9-b0ce-0203b7d455d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:09.396052 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf320d-f3d3-4eb9-b0ce-0203b7d455d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:09.403435 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf320d-f3d3-4eb9-b0ce-0203b7d455d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:09.404911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf320d-f3d3-4eb9-b0ce-0203b7d455d0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:09.418504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-1caf320d-f3d3-4eb9-b0ce-0203b7d455d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:09.418504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:09.418504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:09.418504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:09.418504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:09.418504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:09.418504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:09.418504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:09.418504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:09.418504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:09.418504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:09.418504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:09.418504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:42:09.418504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:42:09.418504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:09.429090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:09.430264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:42:09.430264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:42:09.430264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:42:09.430264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:42:09.430264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:42:09.430264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:42:09.430264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:09.430264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:09.430264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 620/1244] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:09.430930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f0fe5-b74e-447f-b285-7516808bd8e5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:09.557358 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f0fe5-b74e-447f-b285-7516808bd8e5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:09.599418 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2165ac7-bc73-4f27-86c9-d54a0c6ea4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:09.600411 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2165ac7-bc73-4f27-86c9-d54a0c6ea4a2 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:09.600791 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2165ac7-bc73-4f27-86c9-d54a0c6ea4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:09.601166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2165ac7-bc73-4f27-86c9-d54a0c6ea4a2 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:09.602246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2165ac7-bc73-4f27-86c9-d54a0c6ea4a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:09.602763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2165ac7-bc73-4f27-86c9-d54a0c6ea4a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:09.604567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2165ac7-bc73-4f27-86c9-d54a0c6ea4a2 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:09.619268 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 625/1245] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:42:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:09.626316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf320d-f3d3-4eb9-b0ce-0203b7d455d0 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:09.736337 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf320d-f3d3-4eb9-b0ce-0203b7d455d0 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:09.774018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9e70a-efda-4463-9f3b-42e393e20025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:09.774637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9e70a-efda-4463-9f3b-42e393e20025 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:09.774941 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9e70a-efda-4463-9f3b-42e393e20025 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:09.775218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9e70a-efda-4463-9f3b-42e393e20025 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:09.775944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9e70a-efda-4463-9f3b-42e393e20025 None admin] RBAC: Adding query filter params (role.id=44e187abae1d4b248fd145ad9cb97ed2, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=f9506139d5744fe69153e7e30a05b436) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:09.776253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9e70a-efda-4463-9f3b-42e393e20025 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:09.777731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9e70a-efda-4463-9f3b-42e393e20025 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:09.783665 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 621/1246] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:42:09 2022] GET /identity/v3/role_assignments?user.id=f9506139d5744fe69153e7e30a05b436&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=44e187abae1d4b248fd145ad9cb97ed2 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:12.068330 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedbdce-e91c-40ab-8a6a-a64f0ee20793 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:12.068866 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedbdce-e91c-40ab-8a6a-a64f0ee20793 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:12.068990 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedbdce-e91c-40ab-8a6a-a64f0ee20793 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:12.070264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 626/1247] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:12.075988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b370da-e782-489b-834e-7a523cca29bc None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:12.076510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b370da-e782-489b-834e-7a523cca29bc None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:12.076510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b370da-e782-489b-834e-7a523cca29bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:12.082926 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:12.096634 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-72b370da-e782-489b-834e-7a523cca29bc None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:12.098284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-72b370da-e782-489b-834e-7a523cca29bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:12.171697 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 622/1248] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:42:12.449546 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedbdce-e91c-40ab-8a6a-a64f0ee20793 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:12.468402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:12.555575 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedbdce-e91c-40ab-8a6a-a64f0ee20793 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:12.598135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6038f3-07d2-4c18-b639-5e954fe3dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:12.598759 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6038f3-07d2-4c18-b639-5e954fe3dc0b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:12.599072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6038f3-07d2-4c18-b639-5e954fe3dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:12.599346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6038f3-07d2-4c18-b639-5e954fe3dc0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:12.610595 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6038f3-07d2-4c18-b639-5e954fe3dc0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:12.615362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-ba6038f3-07d2-4c18-b639-5e954fe3dc0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:12.615362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:12.615362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:12.615362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:12.615362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:12.615362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:12.615362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:12.615362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:12.615362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:12.615362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:12.615362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:12.615362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:12.615362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:12.615362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:12.615362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:12.615362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:12.616454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:12.616454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:12.616454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:12.616454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:12.616454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:12.616454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:12.616454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:12.616454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:12.616454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:12.616454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:12.616454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:12.616454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:12.616454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:12.616454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:12.616454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:12.616454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:12.617316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:12.617316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:12.617316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:12.617316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:12.617316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:12.617316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:12.617316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:12.617316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:12.617316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:12.617316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:12.617316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:12.617316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:42:12.617316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 627/1249] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:42:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:12.624900 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b370da-e782-489b-834e-7a523cca29bc None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:12.730623 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b370da-e782-489b-834e-7a523cca29bc None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:12.764536 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:12.770085 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af5f93e-eca3-4bbf-9748-6dd34a22001f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:12.770848 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5f93e-eca3-4bbf-9748-6dd34a22001f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:12.771162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5f93e-eca3-4bbf-9748-6dd34a22001f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:12.771430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5f93e-eca3-4bbf-9748-6dd34a22001f None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:12.772092 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af5f93e-eca3-4bbf-9748-6dd34a22001f None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:12.772392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af5f93e-eca3-4bbf-9748-6dd34a22001f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:12.773782 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af5f93e-eca3-4bbf-9748-6dd34a22001f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:12.782605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 623/1250] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:42:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:12.788852 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6038f3-07d2-4c18-b639-5e954fe3dc0b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:12.874176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:12.899208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6038f3-07d2-4c18-b639-5e954fe3dc0b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:12.933253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4f10a-15d8-4790-b4f8-2205c2027937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:12.933876 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4f10a-15d8-4790-b4f8-2205c2027937 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:12.934152 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4f10a-15d8-4790-b4f8-2205c2027937 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:12.934421 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4f10a-15d8-4790-b4f8-2205c2027937 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:12.944752 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4f10a-15d8-4790-b4f8-2205c2027937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:12.946200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4f10a-15d8-4790-b4f8-2205c2027937 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:12.961365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-dae4f10a-15d8-4790-b4f8-2205c2027937 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:12.961365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:12.961365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:12.961365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:12.961365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:12.961365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:12.961365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:12.961365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:12.961365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:12.961365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:12.961365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:12.961365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:12.961365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:12.961365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:12.961365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:12.961365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:12.969204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:12.971858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:12.971858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:12.971858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:42:12.971858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 628/1251] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:12.972292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af5f93e-eca3-4bbf-9748-6dd34a22001f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:13.075814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af5f93e-eca3-4bbf-9748-6dd34a22001f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:13.115596 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bf795-406d-4f43-ab01-c39fc15e87ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:13.116320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bf795-406d-4f43-ab01-c39fc15e87ac None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:13.116446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bf795-406d-4f43-ab01-c39fc15e87ac None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:13.116745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bf795-406d-4f43-ab01-c39fc15e87ac None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:13.117399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bf795-406d-4f43-ab01-c39fc15e87ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:13.117686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bf795-406d-4f43-ab01-c39fc15e87ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:13.119091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bf795-406d-4f43-ab01-c39fc15e87ac None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:13.131904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 624/1252] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:13.137887 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4f10a-15d8-4790-b4f8-2205c2027937 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:13.247257 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4f10a-15d8-4790-b4f8-2205c2027937 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:13.277760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:13.288636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6fcce-d6d0-407a-b0d1-d5ada4b16dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:13.289266 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6fcce-d6d0-407a-b0d1-d5ada4b16dfc None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:13.289547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6fcce-d6d0-407a-b0d1-d5ada4b16dfc None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:13.289943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6fcce-d6d0-407a-b0d1-d5ada4b16dfc None admin] PATH_INFO: `/v3/users/placement` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:13.298789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6fcce-d6d0-407a-b0d1-d5ada4b16dfc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:13.300273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6fcce-d6d0-407a-b0d1-d5ada4b16dfc None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:13.312335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-03e6fcce-d6d0-407a-b0d1-d5ada4b16dfc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:13.312335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:13.312335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:13.312335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:13.312335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:13.312335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:13.312335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:13.312335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:13.312335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:13.312335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:13.312335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:13.312335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:13.312335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:13.312335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:13.312335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:13.312335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:13.315747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:13.316717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:13.316717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:13.316717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:13.316717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:13.316717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:13.316717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:13.316717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:13.316717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:13.316717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:13.316717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:42:13.316717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 629/1253] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:13.321298 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bf795-406d-4f43-ab01-c39fc15e87ac None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:13.384562 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:13.431330 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bf795-406d-4f43-ab01-c39fc15e87ac None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:13.466280 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69192123-2798-4fd9-94e3-34f01bca1290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:13.466916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69192123-2798-4fd9-94e3-34f01bca1290 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:13.467202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69192123-2798-4fd9-94e3-34f01bca1290 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:13.467466 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69192123-2798-4fd9-94e3-34f01bca1290 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:13.468189 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69192123-2798-4fd9-94e3-34f01bca1290 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:13.468475 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69192123-2798-4fd9-94e3-34f01bca1290 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:13.469904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69192123-2798-4fd9-94e3-34f01bca1290 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:13.495433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 625/1254] 10.222.0.91 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:13.502043 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6fcce-d6d0-407a-b0d1-d5ada4b16dfc None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:13.624090 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6fcce-d6d0-407a-b0d1-d5ada4b16dfc None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:13.663767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83526b-3ecf-41cc-b498-6aa5d9460c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:13.664536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83526b-3ecf-41cc-b498-6aa5d9460c95 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:13.664926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83526b-3ecf-41cc-b498-6aa5d9460c95 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:13.665297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83526b-3ecf-41cc-b498-6aa5d9460c95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:13.738773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:13.755710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83526b-3ecf-41cc-b498-6aa5d9460c95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:13.757155 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83526b-3ecf-41cc-b498-6aa5d9460c95 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:13.853018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 630/1255] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:42:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:42:13.859241 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69192123-2798-4fd9-94e3-34f01bca1290 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:13.959017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69192123-2798-4fd9-94e3-34f01bca1290 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:13.973825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:13.995896 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3661e-033a-47d4-a053-6ccbca5bb6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:13.996520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3661e-033a-47d4-a053-6ccbca5bb6b3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:13.996799 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3661e-033a-47d4-a053-6ccbca5bb6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:13.997077 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3661e-033a-47d4-a053-6ccbca5bb6b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:14.007157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3661e-033a-47d4-a053-6ccbca5bb6b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:14.008655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3661e-033a-47d4-a053-6ccbca5bb6b3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:14.023039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-04b3661e-033a-47d4-a053-6ccbca5bb6b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:14.023039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:14.023039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:14.023039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:14.023039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:14.023039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:14.023039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:14.023039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:14.023039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:14.023039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:14.023039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:14.023039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:14.023039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:14.023039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:14.023039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:14.023039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:14.024837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:14.026542 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:14.026542 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:14.026542 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:14.026542 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 626/1256] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:14.029975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83526b-3ecf-41cc-b498-6aa5d9460c95 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:14.117696 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:14.129909 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83526b-3ecf-41cc-b498-6aa5d9460c95 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:14.165699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7961fe7-306c-4cb8-a644-8fbea39e62e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:14.166432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7961fe7-306c-4cb8-a644-8fbea39e62e3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:14.169593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7961fe7-306c-4cb8-a644-8fbea39e62e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:14.169593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7961fe7-306c-4cb8-a644-8fbea39e62e3 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:14.169593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7961fe7-306c-4cb8-a644-8fbea39e62e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:14.169593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7961fe7-306c-4cb8-a644-8fbea39e62e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:14.169593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7961fe7-306c-4cb8-a644-8fbea39e62e3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:14.180841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 631/1257] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:14.186598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3661e-033a-47d4-a053-6ccbca5bb6b3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:14.290241 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3661e-033a-47d4-a053-6ccbca5bb6b3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:14.322559 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab1a5a-4c84-4200-a04f-f9fef044e25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:14.323272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab1a5a-4c84-4200-a04f-f9fef044e25a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:14.323550 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab1a5a-4c84-4200-a04f-f9fef044e25a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:14.323822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab1a5a-4c84-4200-a04f-f9fef044e25a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:14.331038 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab1a5a-4c84-4200-a04f-f9fef044e25a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:14.332529 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab1a5a-4c84-4200-a04f-f9fef044e25a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:14.343938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-afab1a5a-4c84-4200-a04f-f9fef044e25a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:14.343938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:14.343938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:14.343938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:14.343938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:14.343938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:14.343938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:14.343938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:14.343938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:14.343938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:14.343938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:14.343938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:14.343938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:42:14.343938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:42:14.343938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:14.346510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:14.347734 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:42:14.347734 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:42:14.347734 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:42:14.347734 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:42:14.347734 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:42:14.347734 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:42:14.347734 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:14.347734 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:14.347734 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 627/1258] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:42:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:14.351994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7961fe7-306c-4cb8-a644-8fbea39e62e3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:14.453917 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7961fe7-306c-4cb8-a644-8fbea39e62e3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:14.491956 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96032571-de75-48c1-8585-89ba11a3c057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:14.492560 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96032571-de75-48c1-8585-89ba11a3c057 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:14.492838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96032571-de75-48c1-8585-89ba11a3c057 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:14.493113 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96032571-de75-48c1-8585-89ba11a3c057 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:14.493799 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96032571-de75-48c1-8585-89ba11a3c057 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:14.494100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96032571-de75-48c1-8585-89ba11a3c057 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:14.495626 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96032571-de75-48c1-8585-89ba11a3c057 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:14.507443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 632/1259] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:42:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:14.513718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab1a5a-4c84-4200-a04f-f9fef044e25a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:14.542305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:14.615753 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab1a5a-4c84-4200-a04f-f9fef044e25a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:14.651011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaafe3a-be8b-479d-9496-334ff18e7bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:14.651599 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaafe3a-be8b-479d-9496-334ff18e7bd6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:14.651877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaafe3a-be8b-479d-9496-334ff18e7bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:14.652150 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaafe3a-be8b-479d-9496-334ff18e7bd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:14.652860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaafe3a-be8b-479d-9496-334ff18e7bd6 None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=f9506139d5744fe69153e7e30a05b436) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:14.653166 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaafe3a-be8b-479d-9496-334ff18e7bd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:14.654594 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaafe3a-be8b-479d-9496-334ff18e7bd6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:14.659995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 628/1260] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:42:14 2022] GET /identity/v3/role_assignments?user.id=f9506139d5744fe69153e7e30a05b436&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:16.854375 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87fa79-b00f-427f-a3ea-a76b3d915d59 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:16.854975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87fa79-b00f-427f-a3ea-a76b3d915d59 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:16.854975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87fa79-b00f-427f-a3ea-a76b3d915d59 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:16.856311 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 633/1261] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:16.862144 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff9a3e-4fec-4b6a-a9f6-1a1ad9979811 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:16.862546 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff9a3e-4fec-4b6a-a9f6-1a1ad9979811 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:16.862695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff9a3e-4fec-4b6a-a9f6-1a1ad9979811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:16.885186 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-6bff9a3e-4fec-4b6a-a9f6-1a1ad9979811 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:16.886417 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6bff9a3e-4fec-4b6a-a9f6-1a1ad9979811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:16.952162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 629/1262] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:17.226307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87fa79-b00f-427f-a3ea-a76b3d915d59 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:17.269064 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:17.363967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87fa79-b00f-427f-a3ea-a76b3d915d59 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:17.406390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b69a10-77bf-4085-88ca-69d44e38985c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:17.407672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b69a10-77bf-4085-88ca-69d44e38985c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:17.407943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b69a10-77bf-4085-88ca-69d44e38985c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:17.408209 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b69a10-77bf-4085-88ca-69d44e38985c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:17.421796 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b69a10-77bf-4085-88ca-69d44e38985c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:17.426665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-e6b69a10-77bf-4085-88ca-69d44e38985c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:17.426665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:17.426665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:17.426665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:17.426665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:17.426665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:17.426665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:17.426665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:17.426665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:17.426665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:17.426665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:17.426665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:17.426665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:17.426665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:17.426665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:17.426665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:17.428910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:17.428910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:17.428910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:17.428910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:17.428910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:17.428910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:17.428910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:17.428910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:17.428910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:17.428910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:17.428910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:17.428910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:17.428910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:17.428910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:17.428910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:17.428910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:17.430593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:17.430593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:17.430593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:17.430593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:17.430593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:17.430593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:17.430593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:17.430593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:17.430593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:17.430593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:17.430593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:17.430593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:42:17.430593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 634/1263] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:42:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:17.436655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff9a3e-4fec-4b6a-a9f6-1a1ad9979811 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:17.457694 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:17.554931 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff9a3e-4fec-4b6a-a9f6-1a1ad9979811 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:17.599337 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f451b7-43d8-4440-86e9-aad0e627bd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:17.600012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f451b7-43d8-4440-86e9-aad0e627bd36 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:17.600301 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f451b7-43d8-4440-86e9-aad0e627bd36 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:17.600571 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f451b7-43d8-4440-86e9-aad0e627bd36 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:17.601252 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f451b7-43d8-4440-86e9-aad0e627bd36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:17.601686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f451b7-43d8-4440-86e9-aad0e627bd36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:17.603107 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f451b7-43d8-4440-86e9-aad0e627bd36 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:17.611761 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 630/1264] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:42:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:17.618409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b69a10-77bf-4085-88ca-69d44e38985c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:17.701366 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:17.743605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b69a10-77bf-4085-88ca-69d44e38985c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:17.780945 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cbcd0-9b90-4485-aa0e-5d016903b15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:17.783027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cbcd0-9b90-4485-aa0e-5d016903b15e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:17.783027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cbcd0-9b90-4485-aa0e-5d016903b15e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:17.783027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cbcd0-9b90-4485-aa0e-5d016903b15e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:17.792564 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cbcd0-9b90-4485-aa0e-5d016903b15e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:17.794069 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cbcd0-9b90-4485-aa0e-5d016903b15e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:17.808973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-c56cbcd0-9b90-4485-aa0e-5d016903b15e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:17.808973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:17.808973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:17.808973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:17.808973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:17.808973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:17.808973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:17.808973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:17.808973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:17.808973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:17.808973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:17.808973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:17.808973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:17.808973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:17.808973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:17.808973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:17.811138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:17.812805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:17.812805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:17.812805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:42:17.812805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 635/1265] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:17.817935 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f451b7-43d8-4440-86e9-aad0e627bd36 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:17.931561 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:17.951518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f451b7-43d8-4440-86e9-aad0e627bd36 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:17.988353 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdd968-0a79-4a76-a832-726c2014078e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:17.995043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdd968-0a79-4a76-a832-726c2014078e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:17.995043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdd968-0a79-4a76-a832-726c2014078e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:17.995043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdd968-0a79-4a76-a832-726c2014078e None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:17.995043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbdd968-0a79-4a76-a832-726c2014078e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:17.995043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbdd968-0a79-4a76-a832-726c2014078e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:17.995043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbdd968-0a79-4a76-a832-726c2014078e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:18.005820 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 631/1266] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:18.012200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cbcd0-9b90-4485-aa0e-5d016903b15e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:18.153357 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cbcd0-9b90-4485-aa0e-5d016903b15e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:18.188561 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:18.211442 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f7242-87a9-43f5-8493-26b72d618af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:18.212491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f7242-87a9-43f5-8493-26b72d618af8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:18.213144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f7242-87a9-43f5-8493-26b72d618af8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:18.213628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f7242-87a9-43f5-8493-26b72d618af8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:18.227779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f7242-87a9-43f5-8493-26b72d618af8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:18.230511 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3f7242-87a9-43f5-8493-26b72d618af8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:18.252112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-eb3f7242-87a9-43f5-8493-26b72d618af8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:18.252112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:18.252112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:18.252112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:18.252112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:18.252112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:18.252112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:18.252112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:18.252112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:18.252112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:18.252112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:18.252112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:18.252112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:18.252112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:18.252112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:18.252112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:18.253422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:18.254622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:18.254622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:18.254622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:18.254622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:18.254622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:18.254622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:18.254622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:18.254622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:18.254622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:18.254622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:42:18.254622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 636/1267] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:18.261792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdd968-0a79-4a76-a832-726c2014078e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:18.389694 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdd968-0a79-4a76-a832-726c2014078e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:18.426247 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b2e8c-f93b-4131-b248-28b04149121c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:18.426960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b2e8c-f93b-4131-b248-28b04149121c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:18.427234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b2e8c-f93b-4131-b248-28b04149121c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:18.427500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b2e8c-f93b-4131-b248-28b04149121c None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:18.428265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b2e8c-f93b-4131-b248-28b04149121c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:18.428565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b2e8c-f93b-4131-b248-28b04149121c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:18.430002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b2e8c-f93b-4131-b248-28b04149121c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:18.440349 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:18.450007 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 632/1268] 10.222.0.91 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:18.458853 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f7242-87a9-43f5-8493-26b72d618af8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:18.572438 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f7242-87a9-43f5-8493-26b72d618af8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:18.608463 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9603d-3331-4a6b-9d09-4c70c3175ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:18.609092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9603d-3331-4a6b-9d09-4c70c3175ba7 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:18.609373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9603d-3331-4a6b-9d09-4c70c3175ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:18.609644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9603d-3331-4a6b-9d09-4c70c3175ba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:18.620422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9603d-3331-4a6b-9d09-4c70c3175ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:18.621989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9603d-3331-4a6b-9d09-4c70c3175ba7 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:18.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-0ae9603d-3331-4a6b-9d09-4c70c3175ba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:18.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:18.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:18.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:18.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:18.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:18.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:18.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:18.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:18.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:18.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:18.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:18.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:18.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:18.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:18.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:18.639982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:18.642310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:18.642310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:18.642310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:42:18.642310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 637/1269] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:18.646193 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b2e8c-f93b-4131-b248-28b04149121c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:18.759384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b2e8c-f93b-4131-b248-28b04149121c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:18.801171 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89301c19-c203-436a-ad08-e559f3c43090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:18.801775 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89301c19-c203-436a-ad08-e559f3c43090 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:18.802045 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89301c19-c203-436a-ad08-e559f3c43090 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:18.802310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89301c19-c203-436a-ad08-e559f3c43090 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:18.802977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89301c19-c203-436a-ad08-e559f3c43090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:18.803276 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89301c19-c203-436a-ad08-e559f3c43090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:18.804645 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89301c19-c203-436a-ad08-e559f3c43090 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:18.817331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 633/1270] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:18.823367 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9603d-3331-4a6b-9d09-4c70c3175ba7 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:18.858234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:18.941376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9603d-3331-4a6b-9d09-4c70c3175ba7 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:18.976359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f0495-0a6e-47a1-8638-500f1f15d382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:18.976977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f0495-0a6e-47a1-8638-500f1f15d382 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:18.977251 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f0495-0a6e-47a1-8638-500f1f15d382 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:18.977519 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f0495-0a6e-47a1-8638-500f1f15d382 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:18.983993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f0495-0a6e-47a1-8638-500f1f15d382 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:18.985467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f0495-0a6e-47a1-8638-500f1f15d382 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:18.997363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-d51f0495-0a6e-47a1-8638-500f1f15d382 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:18.997363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:18.997363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:18.997363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:18.997363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:18.997363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:18.997363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:18.997363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:18.997363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:18.997363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:18.997363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:18.997363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:18.997363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:42:18.997363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:42:18.997363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:18.999605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:19.000636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:42:19.000636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:42:19.000636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:42:19.000636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:42:19.000636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:42:19.000636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:42:19.000636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:19.000636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:42:19.000636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 638/1271] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:42:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:19.004993 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89301c19-c203-436a-ad08-e559f3c43090 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:19.069313 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:19.115888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89301c19-c203-436a-ad08-e559f3c43090 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:19.153616 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6dce9e-9371-40e8-8203-4b3fa621d82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:19.154239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6dce9e-9371-40e8-8203-4b3fa621d82f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:19.154831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6dce9e-9371-40e8-8203-4b3fa621d82f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:19.154831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6dce9e-9371-40e8-8203-4b3fa621d82f None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:19.155722 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6dce9e-9371-40e8-8203-4b3fa621d82f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:19.155915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6dce9e-9371-40e8-8203-4b3fa621d82f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:19.157393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6dce9e-9371-40e8-8203-4b3fa621d82f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:19.170216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 634/1272] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:42:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:19.178439 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f0495-0a6e-47a1-8638-500f1f15d382 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:19.276064 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:19.284839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f0495-0a6e-47a1-8638-500f1f15d382 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:19.318450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856c6d1-7b21-416a-8eb2-7df37e1342b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:19.319147 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856c6d1-7b21-416a-8eb2-7df37e1342b4 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:19.319645 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856c6d1-7b21-416a-8eb2-7df37e1342b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:19.319872 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856c6d1-7b21-416a-8eb2-7df37e1342b4 None admin] PATH_INFO: `/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/f9506139d5744fe69153e7e30a05b436/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:19.321556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856c6d1-7b21-416a-8eb2-7df37e1342b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb2bd53d40841eeb1653bd5927fb189, user_id=f9506139d5744fe69153e7e30a05b436, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:19.323003 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856c6d1-7b21-416a-8eb2-7df37e1342b4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:19.424824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 639/1273] 10.222.0.91 () {58 vars in 1439 bytes} [Thu Sep 8 13:42:19 2022] PUT /identity/v3/projects/3fb2bd53d40841eeb1653bd5927fb189/users/f9506139d5744fe69153e7e30a05b436/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:42:21.714280 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d7c17-989c-474e-86a6-def74acf525d None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:21.714835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d7c17-989c-474e-86a6-def74acf525d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:21.714835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d7c17-989c-474e-86a6-def74acf525d None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:21.716252 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 635/1274] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:21.722755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734fb17-9188-4693-8088-cb65f4fc5f71 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:21.723803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734fb17-9188-4693-8088-cb65f4fc5f71 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:21.723803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734fb17-9188-4693-8088-cb65f4fc5f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:21.745174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3734fb17-9188-4693-8088-cb65f4fc5f71 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:21.746700 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3734fb17-9188-4693-8088-cb65f4fc5f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:21.828658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 640/1275] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:22.100406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d7c17-989c-474e-86a6-def74acf525d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:22.230234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d7c17-989c-474e-86a6-def74acf525d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:22.248538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:22.270022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c7bae-6332-4892-b348-564c24d798e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:22.270792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c7bae-6332-4892-b348-564c24d798e3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:22.271106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c7bae-6332-4892-b348-564c24d798e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:22.271375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c7bae-6332-4892-b348-564c24d798e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:22.281484 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c7bae-6332-4892-b348-564c24d798e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:22.286327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-c73c7bae-6332-4892-b348-564c24d798e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:22.286327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:22.286327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:22.286327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:22.286327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:22.286327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:22.286327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:22.286327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:22.286327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:22.286327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:22.286327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:22.286327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:22.286327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:22.286327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:22.286327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:22.286327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:22.288238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:22.288238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:22.288238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:22.288238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:22.288238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:22.288238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:22.288238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:22.288238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:22.288238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:22.288238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:22.288238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:22.288238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:22.288238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:22.288238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:22.288238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:22.288238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:22.289649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:22.289649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:22.289649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:22.289649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:22.289649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:22.289649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:22.289649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:22.289649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:22.289649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:22.289649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:22.289649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:22.289649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:22.289649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 636/1276] 10.222.0.91 () {56 vars in 1096 bytes} [Thu Sep 8 13:42:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:22.302969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734fb17-9188-4693-8088-cb65f4fc5f71 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:22.393747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:22.421598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734fb17-9188-4693-8088-cb65f4fc5f71 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:22.458974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17db454-bb6f-48c2-b143-1ee7b530fafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:22.459588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17db454-bb6f-48c2-b143-1ee7b530fafe None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:22.459859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17db454-bb6f-48c2-b143-1ee7b530fafe None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:22.460126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17db454-bb6f-48c2-b143-1ee7b530fafe None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:22.460829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17db454-bb6f-48c2-b143-1ee7b530fafe None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:22.461124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17db454-bb6f-48c2-b143-1ee7b530fafe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:22.462487 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17db454-bb6f-48c2-b143-1ee7b530fafe None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:22.471362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 641/1277] 10.222.0.91 () {56 vars in 1099 bytes} [Thu Sep 8 13:42:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:22.477842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c7bae-6332-4892-b348-564c24d798e3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:22.601770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c7bae-6332-4892-b348-564c24d798e3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:22.640760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e212ee-c8de-469c-a173-bc65d5154007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:22.641397 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e212ee-c8de-469c-a173-bc65d5154007 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:22.641674 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e212ee-c8de-469c-a173-bc65d5154007 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:22.641948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e212ee-c8de-469c-a173-bc65d5154007 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:22.653085 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e212ee-c8de-469c-a173-bc65d5154007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:22.654479 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e212ee-c8de-469c-a173-bc65d5154007 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:22.671138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-30e212ee-c8de-469c-a173-bc65d5154007 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:22.671138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:22.671138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:22.671138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:22.671138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:22.671138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:22.671138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:22.671138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:22.671138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:22.671138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:22.671138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:22.671138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:22.671138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:22.671138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:22.671138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:22.671138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:22.672207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:22.673160 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:22.673160 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:22.673160 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:22.673160 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 637/1278] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:22.678933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17db454-bb6f-48c2-b143-1ee7b530fafe None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:22.801116 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17db454-bb6f-48c2-b143-1ee7b530fafe None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:22.838508 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9124298-482b-476d-8529-8dff7d62be9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:22.839152 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9124298-482b-476d-8529-8dff7d62be9b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:22.839432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9124298-482b-476d-8529-8dff7d62be9b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:22.839707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9124298-482b-476d-8529-8dff7d62be9b None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:22.840430 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9124298-482b-476d-8529-8dff7d62be9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:22.840736 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9124298-482b-476d-8529-8dff7d62be9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:22.842207 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9124298-482b-476d-8529-8dff7d62be9b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:22.856721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 642/1279] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:22.863742 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e212ee-c8de-469c-a173-bc65d5154007 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:22.903064 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:22.978185 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e212ee-c8de-469c-a173-bc65d5154007 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:23.041421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e5bda-419e-474b-bb7e-97d24644d95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:23.042640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e5bda-419e-474b-bb7e-97d24644d95f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:23.043267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e5bda-419e-474b-bb7e-97d24644d95f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:23.043780 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e5bda-419e-474b-bb7e-97d24644d95f None admin] PATH_INFO: `/v3/users/placement` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:23.059741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e5bda-419e-474b-bb7e-97d24644d95f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:23.062314 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e5bda-419e-474b-bb7e-97d24644d95f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:23.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-b66e5bda-419e-474b-bb7e-97d24644d95f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:23.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:23.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:23.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:23.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:23.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:23.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:23.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:23.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:23.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:23.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:23.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:23.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:23.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:23.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:23.081679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:23.083396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:23.084587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:23.084587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:23.084587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:23.084587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:23.084587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:23.084587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:23.084587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:23.084587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:23.084587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:23.084587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:23.085534 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 638/1280] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:23.094732 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9124298-482b-476d-8529-8dff7d62be9b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:23.101425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:23.256448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9124298-482b-476d-8529-8dff7d62be9b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:23.294709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29c33d-e03a-40c1-9b3e-cce4c755635b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:23.295393 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29c33d-e03a-40c1-9b3e-cce4c755635b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:23.295675 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29c33d-e03a-40c1-9b3e-cce4c755635b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:23.295948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29c33d-e03a-40c1-9b3e-cce4c755635b None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:23.296671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29c33d-e03a-40c1-9b3e-cce4c755635b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:23.296978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29c33d-e03a-40c1-9b3e-cce4c755635b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:23.298430 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c29c33d-e03a-40c1-9b3e-cce4c755635b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:23.318112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 643/1281] 10.222.0.91 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:23.324553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e5bda-419e-474b-bb7e-97d24644d95f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:23.444957 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:23.455623 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e5bda-419e-474b-bb7e-97d24644d95f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:23.497593 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1f2d7-7978-49d3-9cbf-74d4d344085d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:23.498210 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1f2d7-7978-49d3-9cbf-74d4d344085d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:23.498487 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1f2d7-7978-49d3-9cbf-74d4d344085d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:23.498762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1f2d7-7978-49d3-9cbf-74d4d344085d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:23.565957 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1f2d7-7978-49d3-9cbf-74d4d344085d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:23.567418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1f2d7-7978-49d3-9cbf-74d4d344085d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:23.670015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:23.680546 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 639/1282] 10.222.0.91 () {58 vars in 1282 bytes} [Thu Sep 8 13:42:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 13:42:23.687021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29c33d-e03a-40c1-9b3e-cce4c755635b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:23.753727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:23.792428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29c33d-e03a-40c1-9b3e-cce4c755635b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:23.841100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda894e-5f6f-411d-ad78-b3202fd76e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:23.841981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda894e-5f6f-411d-ad78-b3202fd76e35 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:23.842261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda894e-5f6f-411d-ad78-b3202fd76e35 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:23.842531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda894e-5f6f-411d-ad78-b3202fd76e35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:23.855383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda894e-5f6f-411d-ad78-b3202fd76e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:23.856895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda894e-5f6f-411d-ad78-b3202fd76e35 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:23.872623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-cfda894e-5f6f-411d-ad78-b3202fd76e35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:23.872623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:23.872623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:23.872623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:23.872623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:23.872623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:23.872623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:23.872623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:23.872623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:23.872623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:23.872623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:23.872623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:23.872623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:23.872623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:23.872623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:23.872623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:23.878817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:23.879927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:23.879927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:23.879927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:42:23.879927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 644/1283] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:23.881118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1f2d7-7978-49d3-9cbf-74d4d344085d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:23.989658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1f2d7-7978-49d3-9cbf-74d4d344085d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:24.030137 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddffbd8-347b-46f7-bb8e-f3eaef015642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:24.030750 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddffbd8-347b-46f7-bb8e-f3eaef015642 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:24.031060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddffbd8-347b-46f7-bb8e-f3eaef015642 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:24.031331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddffbd8-347b-46f7-bb8e-f3eaef015642 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:24.031970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddffbd8-347b-46f7-bb8e-f3eaef015642 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:24.032272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddffbd8-347b-46f7-bb8e-f3eaef015642 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:24.033653 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddffbd8-347b-46f7-bb8e-f3eaef015642 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:24.045403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 640/1284] 10.222.0.91 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:24.051362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda894e-5f6f-411d-ad78-b3202fd76e35 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:24.160864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda894e-5f6f-411d-ad78-b3202fd76e35 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:24.175024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:24.196287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ac11d-60c4-4282-9a93-7f84c1510853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:24.197137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ac11d-60c4-4282-9a93-7f84c1510853 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:24.197359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ac11d-60c4-4282-9a93-7f84c1510853 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:24.197692 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ac11d-60c4-4282-9a93-7f84c1510853 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:24.204909 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ac11d-60c4-4282-9a93-7f84c1510853 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:24.206456 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ac11d-60c4-4282-9a93-7f84c1510853 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:24.218978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-388ac11d-60c4-4282-9a93-7f84c1510853 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:24.218978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:24.218978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:24.218978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:24.218978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:24.218978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:24.218978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:24.218978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:24.218978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:24.218978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:24.218978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:24.218978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:24.218978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:42:24.218978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:42:24.218978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:24.220112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:24.221586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:42:24.221586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:42:24.221586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:42:24.221586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:42:24.221586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:42:24.221586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:42:24.221586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:24.221586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Sep 08 13:42:24.221586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 645/1285] 10.222.0.91 () {56 vars in 1111 bytes} [Thu Sep 8 13:42:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:24.227001 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddffbd8-347b-46f7-bb8e-f3eaef015642 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:24.281834 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:24.339833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddffbd8-347b-46f7-bb8e-f3eaef015642 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:24.380590 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4587e-6e95-4179-a797-f748ba902665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:24.381262 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4587e-6e95-4179-a797-f748ba902665 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:24.381532 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4587e-6e95-4179-a797-f748ba902665 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:24.381799 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4587e-6e95-4179-a797-f748ba902665 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:24.382582 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4587e-6e95-4179-a797-f748ba902665 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:24.382939 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4587e-6e95-4179-a797-f748ba902665 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:24.384396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4587e-6e95-4179-a797-f748ba902665 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:24.397642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 641/1286] 10.222.0.91 () {56 vars in 1148 bytes} [Thu Sep 8 13:42:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:24.404074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ac11d-60c4-4282-9a93-7f84c1510853 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:24.509904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ac11d-60c4-4282-9a93-7f84c1510853 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:24.555154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342acd1-35fe-4137-b3d4-38d7f1237080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:24.556711 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342acd1-35fe-4137-b3d4-38d7f1237080 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:24.557550 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342acd1-35fe-4137-b3d4-38d7f1237080 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:24.558460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342acd1-35fe-4137-b3d4-38d7f1237080 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:24.561269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342acd1-35fe-4137-b3d4-38d7f1237080 None admin] RBAC: Adding query filter params (role.id=32d91beb67f2452c995f72173bb8ea4f, scope.project.id=3fb2bd53d40841eeb1653bd5927fb189, user.id=f9506139d5744fe69153e7e30a05b436) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:24.564609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342acd1-35fe-4137-b3d4-38d7f1237080 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:24.564609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342acd1-35fe-4137-b3d4-38d7f1237080 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:24.569134 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 646/1287] 10.222.0.91 () {56 vars in 1374 bytes} [Thu Sep 8 13:42:24 2022] GET /identity/v3/role_assignments?user.id=f9506139d5744fe69153e7e30a05b436&scope.project.id=3fb2bd53d40841eeb1653bd5927fb189&role.id=32d91beb67f2452c995f72173bb8ea4f => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:26.807945 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3fc81-541b-4f2f-9600-5b39ec6f67dc None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:26.808464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3fc81-541b-4f2f-9600-5b39ec6f67dc None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:26.808464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3fc81-541b-4f2f-9600-5b39ec6f67dc None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:26.809791 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 642/1288] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:26.815148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2b19d-1108-4a6f-a719-a03477718bd2 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:26.815702 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2b19d-1108-4a6f-a719-a03477718bd2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:26.815702 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2b19d-1108-4a6f-a719-a03477718bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:26.833693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ffb2b19d-1108-4a6f-a719-a03477718bd2 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:26.834866 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ffb2b19d-1108-4a6f-a719-a03477718bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:26.864375 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:26.909188 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 647/1289] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:42:27.192825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3fc81-541b-4f2f-9600-5b39ec6f67dc None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:27.303301 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3fc81-541b-4f2f-9600-5b39ec6f67dc None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:27.314526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:27.342554 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff1041-1a51-4c43-a993-05e627a67301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:27.343338 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff1041-1a51-4c43-a993-05e627a67301 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:27.343633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff1041-1a51-4c43-a993-05e627a67301 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:27.343905 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff1041-1a51-4c43-a993-05e627a67301 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:27.344560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff1041-1a51-4c43-a993-05e627a67301 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:27.345940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff1041-1a51-4c43-a993-05e627a67301 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-71ff1041-1a51-4c43-a993-05e627a67301 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:27.354573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:27.355666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:27.355666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:27.355666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:27.355666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:27.355666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:27.355666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:27.355666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:42:27.355666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:42:27.355666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:42:27.355666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:42:27.355666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:27.355666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:27.355666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:27.355666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:27.355666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:27.355666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:27.357065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:27.357065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:27.357065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:27.357065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:27.357065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:27.357065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:27.357065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:27.357065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:27.357065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:27.357065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:27.357065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:42:27.357065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:42:27.357065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:42:27.357065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:27.357065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 643/1290] 10.222.0.91 () {56 vars in 1117 bytes} [Thu Sep 8 13:42:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:27.362652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2b19d-1108-4a6f-a719-a03477718bd2 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:27.491209 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2b19d-1108-4a6f-a719-a03477718bd2 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:27.531961 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04732f-1db1-4cf2-8a20-a3ec67d7cde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:27.532582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04732f-1db1-4cf2-8a20-a3ec67d7cde5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:27.532865 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04732f-1db1-4cf2-8a20-a3ec67d7cde5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:27.533135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04732f-1db1-4cf2-8a20-a3ec67d7cde5 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:27.533767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04732f-1db1-4cf2-8a20-a3ec67d7cde5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:27.534062 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04732f-1db1-4cf2-8a20-a3ec67d7cde5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:27.535495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04732f-1db1-4cf2-8a20-a3ec67d7cde5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:27.540663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 648/1291] 10.222.0.91 () {56 vars in 1116 bytes} [Thu Sep 8 13:42:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:27.547412 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff1041-1a51-4c43-a993-05e627a67301 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:27.651670 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff1041-1a51-4c43-a993-05e627a67301 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:27.687175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7ee93-8f9e-4c55-b628-7e54c43e0354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:27.687802 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7ee93-8f9e-4c55-b628-7e54c43e0354 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:27.688183 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7ee93-8f9e-4c55-b628-7e54c43e0354 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:27.688434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7ee93-8f9e-4c55-b628-7e54c43e0354 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:27.689056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7ee93-8f9e-4c55-b628-7e54c43e0354 None admin] RBAC: Adding query filter params (type=placement) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:27.689369 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7ee93-8f9e-4c55-b628-7e54c43e0354 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:27.690806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7ee93-8f9e-4c55-b628-7e54c43e0354 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:27.697058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 644/1292] 10.222.0.91 () {56 vars in 1116 bytes} [Thu Sep 8 13:42:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:29.881602 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa1f45-3932-4b48-ba3b-8019ac58b7ef None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:29.882195 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa1f45-3932-4b48-ba3b-8019ac58b7ef None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:29.882195 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa1f45-3932-4b48-ba3b-8019ac58b7ef None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:29.883556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 649/1293] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:29.889390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed0d2e-0e39-43a3-a08e-97952144e9cf None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:29.889946 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed0d2e-0e39-43a3-a08e-97952144e9cf None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:29.889946 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed0d2e-0e39-43a3-a08e-97952144e9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:29.908977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b1ed0d2e-0e39-43a3-a08e-97952144e9cf None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:29.910178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b1ed0d2e-0e39-43a3-a08e-97952144e9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:29.941866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:29.983282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 645/1294] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:30.270231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa1f45-3932-4b48-ba3b-8019ac58b7ef None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:30.302398 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:30.375967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa1f45-3932-4b48-ba3b-8019ac58b7ef None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:30.411599 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958a49b1-343b-46d2-b33a-80531d328be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:30.412256 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a49b1-343b-46d2-b33a-80531d328be1 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:30.412531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a49b1-343b-46d2-b33a-80531d328be1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:30.412803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958a49b1-343b-46d2-b33a-80531d328be1 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:30.413445 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a49b1-343b-46d2-b33a-80531d328be1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:30.414807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958a49b1-343b-46d2-b33a-80531d328be1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:30.626282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 650/1295] 10.222.0.91 () {60 vars in 1141 bytes} [Thu Sep 8 13:42:30 2022] POST /identity/v3/services => generated 246 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:42:32.836669 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34354149-b85b-4c12-b88b-64880d0a7a9d None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:32.837251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34354149-b85b-4c12-b88b-64880d0a7a9d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:32.837251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34354149-b85b-4c12-b88b-64880d0a7a9d None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:32.838524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 646/1296] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:32.844577 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7165a2f-97ac-4973-8bf8-1b2e4375658a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:32.845058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7165a2f-97ac-4973-8bf8-1b2e4375658a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:32.845186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7165a2f-97ac-4973-8bf8-1b2e4375658a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:32.864401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e7165a2f-97ac-4973-8bf8-1b2e4375658a None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:32.865610 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e7165a2f-97ac-4973-8bf8-1b2e4375658a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:32.941833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 651/1297] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:32 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:33.222869 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34354149-b85b-4c12-b88b-64880d0a7a9d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:33.327952 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34354149-b85b-4c12-b88b-64880d0a7a9d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:33.362366 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62142bae-e5c1-406a-b951-9e753c87c6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:33.363040 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62142bae-e5c1-406a-b951-9e753c87c6bc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:33.363318 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62142bae-e5c1-406a-b951-9e753c87c6bc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:33.363565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62142bae-e5c1-406a-b951-9e753c87c6bc None admin] PATH_INFO: `/v3/services/placement` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:33.364136 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62142bae-e5c1-406a-b951-9e753c87c6bc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:33.365493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62142bae-e5c1-406a-b951-9e753c87c6bc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-62142bae-e5c1-406a-b951-9e753c87c6bc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:33.374096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:33.375350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:33.375350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:33.375350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:33.375350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:33.375350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:33.375350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:33.375350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:42:33.375350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:42:33.375350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:42:33.375350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:42:33.375350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:33.375350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:33.375350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:33.375350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:33.375350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:33.375350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:33.376419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:33.376419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:33.376419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:33.376419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:33.376419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:33.376419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:33.376419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:33.376419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:33.376419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:33.376419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:33.376419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:42:33.376419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:42:33.376419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:42:33.376419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:33.376419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 647/1298] 10.222.0.91 () {56 vars in 1117 bytes} [Thu Sep 8 13:42:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:33.383292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7165a2f-97ac-4973-8bf8-1b2e4375658a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:33.412195 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:33.499037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7165a2f-97ac-4973-8bf8-1b2e4375658a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:33.534393 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a0a3e-0f17-479b-9dc3-3812eb87647e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:33.535027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a0a3e-0f17-479b-9dc3-3812eb87647e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:33.535305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a0a3e-0f17-479b-9dc3-3812eb87647e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:33.535568 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a0a3e-0f17-479b-9dc3-3812eb87647e None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:33.536186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a0a3e-0f17-479b-9dc3-3812eb87647e None admin] RBAC: Adding query filter params (name=placement) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:33.536562 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a0a3e-0f17-479b-9dc3-3812eb87647e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:33.537863 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a0a3e-0f17-479b-9dc3-3812eb87647e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:33.543107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 652/1299] 10.222.0.91 () {56 vars in 1116 bytes} [Thu Sep 8 13:42:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:33.549264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62142bae-e5c1-406a-b951-9e753c87c6bc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:33.585174 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:33.654489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62142bae-e5c1-406a-b951-9e753c87c6bc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:33.688297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e849f93-4eaa-44c8-b421-be0d29c0ef97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:33.688897 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e849f93-4eaa-44c8-b421-be0d29c0ef97 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:33.689165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e849f93-4eaa-44c8-b421-be0d29c0ef97 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:33.689437 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e849f93-4eaa-44c8-b421-be0d29c0ef97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:33.690122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e849f93-4eaa-44c8-b421-be0d29c0ef97 None admin] RBAC: Adding query filter params (interface=public, service_id=6400482f98884092a2ea29d1839d3fcf, region_id=RegionOne) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:33.690488 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e849f93-4eaa-44c8-b421-be0d29c0ef97 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:33.691876 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e849f93-4eaa-44c8-b421-be0d29c0ef97 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:33.697705 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 648/1300] 10.222.0.91 () {56 vars in 1251 bytes} [Thu Sep 8 13:42:33 2022] GET /identity/v3/endpoints?service_id=6400482f98884092a2ea29d1839d3fcf&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:33.703573 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a0a3e-0f17-479b-9dc3-3812eb87647e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:33.787974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:33.806601 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a0a3e-0f17-479b-9dc3-3812eb87647e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:33.842638 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8e55c-a7b5-4f73-af7d-494f976ba93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:33.842638 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8e55c-a7b5-4f73-af7d-494f976ba93c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:33.845797 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8e55c-a7b5-4f73-af7d-494f976ba93c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:33.845797 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8e55c-a7b5-4f73-af7d-494f976ba93c None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:33.845797 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8e55c-a7b5-4f73-af7d-494f976ba93c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:33.845797 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8e55c-a7b5-4f73-af7d-494f976ba93c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:33.852580 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 653/1301] 10.222.0.91 () {56 vars in 1087 bytes} [Thu Sep 8 13:42:33 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:42:36.034303 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7522ad-00e0-4172-9c55-c8ebd3a0c284 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:36.034845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7522ad-00e0-4172-9c55-c8ebd3a0c284 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:36.034845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7522ad-00e0-4172-9c55-c8ebd3a0c284 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:36.036209 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 649/1302] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:36.042528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52a806-d27d-4df9-a5c2-cd39a9b74e7a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:36.043055 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52a806-d27d-4df9-a5c2-cd39a9b74e7a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:36.043166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52a806-d27d-4df9-a5c2-cd39a9b74e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:36.061636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-bd52a806-d27d-4df9-a5c2-cd39a9b74e7a None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:36.063028 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-bd52a806-d27d-4df9-a5c2-cd39a9b74e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:36.131533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 654/1303] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:36 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:36.403595 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7522ad-00e0-4172-9c55-c8ebd3a0c284 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:36.503136 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:36.510035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7522ad-00e0-4172-9c55-c8ebd3a0c284 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:36.543183 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eecf80-5e7e-4063-b631-695070843e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:36.543801 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eecf80-5e7e-4063-b631-695070843e88 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:36.544081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eecf80-5e7e-4063-b631-695070843e88 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:36.544354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eecf80-5e7e-4063-b631-695070843e88 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:36.544929 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eecf80-5e7e-4063-b631-695070843e88 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:36.546689 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eecf80-5e7e-4063-b631-695070843e88 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-36eecf80-5e7e-4063-b631-695070843e88 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:36.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:36.556933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:36.556933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:36.556933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:36.556933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:36.556933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:36.556933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:36.556933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:42:36.556933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:42:36.556933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:42:36.556933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:42:36.556933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:36.556933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:36.556933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:36.556933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:36.556933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:36.556933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:36.558290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:36.558290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:36.558290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:36.558290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:36.558290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:36.558290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:36.558290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:36.558290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:36.558290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:36.558290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:36.558290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:42:36.558290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:42:36.558290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:42:36.558290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Sep 08 13:42:36.558290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 650/1304] 10.222.0.91 () {56 vars in 1117 bytes} [Thu Sep 8 13:42:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:36.563830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52a806-d27d-4df9-a5c2-cd39a9b74e7a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:36.642806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:36.666354 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52a806-d27d-4df9-a5c2-cd39a9b74e7a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:36.703082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c7a33-ae86-44ab-b573-2f3133d3fa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:36.703688 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c7a33-ae86-44ab-b573-2f3133d3fa43 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:36.703968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c7a33-ae86-44ab-b573-2f3133d3fa43 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:36.705896 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c7a33-ae86-44ab-b573-2f3133d3fa43 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:36.705896 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c7a33-ae86-44ab-b573-2f3133d3fa43 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:36.705896 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c7a33-ae86-44ab-b573-2f3133d3fa43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:36.706610 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c7a33-ae86-44ab-b573-2f3133d3fa43 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:36.712341 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 655/1305] 10.222.0.91 () {56 vars in 1116 bytes} [Thu Sep 8 13:42:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:36.718701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eecf80-5e7e-4063-b631-695070843e88 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:36.818042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eecf80-5e7e-4063-b631-695070843e88 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:36.852507 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e407c7-fcf7-48cb-a6f2-fa2609fa9202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:36.853106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e407c7-fcf7-48cb-a6f2-fa2609fa9202 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:36.853387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e407c7-fcf7-48cb-a6f2-fa2609fa9202 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:36.860581 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e407c7-fcf7-48cb-a6f2-fa2609fa9202 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:36.860581 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e407c7-fcf7-48cb-a6f2-fa2609fa9202 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:36.860581 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e407c7-fcf7-48cb-a6f2-fa2609fa9202 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:36.972258 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 651/1306] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:42:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:42:46.708720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=30 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:46.966845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:03.950376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40589832-4ece-4468-a9fb-06c6f4dd67c9 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:03.951046 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40589832-4ece-4468-a9fb-06c6f4dd67c9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:03.951826 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40589832-4ece-4468-a9fb-06c6f4dd67c9 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:03.954482 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 656/1307] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:51:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:03.960309 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2ac62-e83b-4636-98c5-94202e8667f1 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:03.961912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2ac62-e83b-4636-98c5-94202e8667f1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:03.961912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2ac62-e83b-4636-98c5-94202e8667f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:04.061522 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-34f2ac62-e83b-4636-98c5-94202e8667f1 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:04.063172 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-34f2ac62-e83b-4636-98c5-94202e8667f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:04.214988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 652/1308] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:04.530558 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f689cd-1125-4511-adce-8e4589a3ee34 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:04.531209 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f689cd-1125-4511-adce-8e4589a3ee34 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:04.531840 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f689cd-1125-4511-adce-8e4589a3ee34 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:04.533847 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 657/1309] 10.222.0.91 () {54 vars in 965 bytes} [Thu Sep 8 13:51:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:04.545594 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa1cc5-6715-4f18-8011-37ad67772d66 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:04.546049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa1cc5-6715-4f18-8011-37ad67772d66 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:04.546435 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa1cc5-6715-4f18-8011-37ad67772d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:04.670557 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-98fa1cc5-6715-4f18-8011-37ad67772d66 None None] MFA Rules not processed for user `cdbbaa64ad4844a0b2698dee82f95f23`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:04.672228 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-98fa1cc5-6715-4f18-8011-37ad67772d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:04.712198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:04.786211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 653/1310] 10.222.0.91 () {58 vars in 1062 bytes} [Thu Sep 8 13:51:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:04.798329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f689cd-1125-4511-adce-8e4589a3ee34 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:04.803988 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b2f689cd-1125-4511-adce-8e4589a3ee34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:04.975125 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f689cd-1125-4511-adce-8e4589a3ee34 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:05.096526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d0d97-5ad7-44ae-9616-0358b35c18ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:05.097162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d0d97-5ad7-44ae-9616-0358b35c18ac service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:05.097585 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d0d97-5ad7-44ae-9616-0358b35c18ac service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:05.098159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d0d97-5ad7-44ae-9616-0358b35c18ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:05.101111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-507d0d97-5ad7-44ae-9616-0358b35c18ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:05.188141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:05.270259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d0d97-5ad7-44ae-9616-0358b35c18ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:05.272869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d0d97-5ad7-44ae-9616-0358b35c18ac service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:05.304491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 658/1311] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:07.861833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654a73a-966f-443f-b1ee-2bcad05d6f59 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:07.862375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654a73a-966f-443f-b1ee-2bcad05d6f59 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:07.862375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654a73a-966f-443f-b1ee-2bcad05d6f59 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:07.863888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 654/1312] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:07.869751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653570ae-9aca-4643-9b01-aceabdb75b39 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:07.870258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653570ae-9aca-4643-9b01-aceabdb75b39 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:07.870379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653570ae-9aca-4643-9b01-aceabdb75b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:07.945949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-653570ae-9aca-4643-9b01-aceabdb75b39 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:07.947203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-653570ae-9aca-4643-9b01-aceabdb75b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:08.071069 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 659/1313] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:08.379163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6beb73-fb80-406a-be7f-81744d8c2a4f None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:08.379453 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6beb73-fb80-406a-be7f-81744d8c2a4f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:08.379837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6beb73-fb80-406a-be7f-81744d8c2a4f None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:08.381200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 655/1314] 10.222.0.91 () {54 vars in 965 bytes} [Thu Sep 8 13:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:08.392986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2549c55-30c2-4a1d-8db0-44c5c2aae898 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:08.393405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2549c55-30c2-4a1d-8db0-44c5c2aae898 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:08.393962 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2549c55-30c2-4a1d-8db0-44c5c2aae898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:08.408303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:08.484611 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e2549c55-30c2-4a1d-8db0-44c5c2aae898 None None] MFA Rules not processed for user `cdbbaa64ad4844a0b2698dee82f95f23`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:08.486079 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e2549c55-30c2-4a1d-8db0-44c5c2aae898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:08.577307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 660/1315] 10.222.0.91 () {58 vars in 1062 bytes} [Thu Sep 8 13:51:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:08.587351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6beb73-fb80-406a-be7f-81744d8c2a4f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:08.589017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5e6beb73-fb80-406a-be7f-81744d8c2a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:08.719735 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6beb73-fb80-406a-be7f-81744d8c2a4f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:08.804473 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:08.809054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb0ef5-0838-406c-9312-7a8ed4c19401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:08.809912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb0ef5-0838-406c-9312-7a8ed4c19401 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:08.810358 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb0ef5-0838-406c-9312-7a8ed4c19401 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:08.810695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb0ef5-0838-406c-9312-7a8ed4c19401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:08.813604 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-79fb0ef5-0838-406c-9312-7a8ed4c19401 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:08.938325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb0ef5-0838-406c-9312-7a8ed4c19401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:08.939954 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb0ef5-0838-406c-9312-7a8ed4c19401 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:08.962190 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 656/1316] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:12.149623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689ae77-86ba-4f5b-818f-06a73c04c244 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:12.150218 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689ae77-86ba-4f5b-818f-06a73c04c244 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:12.150218 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689ae77-86ba-4f5b-818f-06a73c04c244 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:12.152125 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 661/1317] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:51:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:12.158612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68dc6a-d6c8-49d7-91f6-b9b606762bdd None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:12.159458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68dc6a-d6c8-49d7-91f6-b9b606762bdd None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:12.159458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68dc6a-d6c8-49d7-91f6-b9b606762bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:12.249360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9b68dc6a-d6c8-49d7-91f6-b9b606762bdd None None] MFA Rules not processed for user `9f12e9e4aad240d09cdbf43f68f765de`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:12.251499 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9b68dc6a-d6c8-49d7-91f6-b9b606762bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:12.317804 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:12.366628 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 657/1318] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:51:12 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:12.660299 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689ae77-86ba-4f5b-818f-06a73c04c244 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:12.661937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5689ae77-86ba-4f5b-818f-06a73c04c244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:12.781722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689ae77-86ba-4f5b-818f-06a73c04c244 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:12.784578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:12.865498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393e6ce-a77f-4b69-af2e-fbfcca0cbf53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:12.866110 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393e6ce-a77f-4b69-af2e-fbfcca0cbf53 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:12.866389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393e6ce-a77f-4b69-af2e-fbfcca0cbf53 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:12.866660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393e6ce-a77f-4b69-af2e-fbfcca0cbf53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:12.868351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3393e6ce-a77f-4b69-af2e-fbfcca0cbf53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:12.997666 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393e6ce-a77f-4b69-af2e-fbfcca0cbf53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:12.999405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393e6ce-a77f-4b69-af2e-fbfcca0cbf53 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:13.018939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 662/1319] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:12 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:51:16.595645 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face3897-8698-4fc8-9288-004fc5c1d6b4 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:16.596182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face3897-8698-4fc8-9288-004fc5c1d6b4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:16.596182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face3897-8698-4fc8-9288-004fc5c1d6b4 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:16.597621 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 658/1320] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:16.603065 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e490-42cd-4d39-9b5d-6cd611b909ed None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:16.603509 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e490-42cd-4d39-9b5d-6cd611b909ed None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:16.603647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e490-42cd-4d39-9b5d-6cd611b909ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:16.656816 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:16.684450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d6b5e490-42cd-4d39-9b5d-6cd611b909ed None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:16.685677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d6b5e490-42cd-4d39-9b5d-6cd611b909ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:16.805228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 663/1321] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:51:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:17.101006 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face3897-8698-4fc8-9288-004fc5c1d6b4 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:17.102537 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-face3897-8698-4fc8-9288-004fc5c1d6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:17.209709 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face3897-8698-4fc8-9288-004fc5c1d6b4 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:17.292058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190013e9-ff76-472e-be99-81a99ae5a83d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:17.292654 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190013e9-ff76-472e-be99-81a99ae5a83d service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:17.292966 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190013e9-ff76-472e-be99-81a99ae5a83d service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:17.293238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190013e9-ff76-472e-be99-81a99ae5a83d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:17.294868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-190013e9-ff76-472e-be99-81a99ae5a83d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:17.302439 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:17.413197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190013e9-ff76-472e-be99-81a99ae5a83d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:17.414621 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190013e9-ff76-472e-be99-81a99ae5a83d service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:17.433034 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 659/1322] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:20.974503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539516d7-21d0-42ff-84ad-3d937c06f30f None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:20.975225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539516d7-21d0-42ff-84ad-3d937c06f30f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:20.975400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539516d7-21d0-42ff-84ad-3d937c06f30f None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:20.976784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 664/1323] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:51:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:20.984055 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8598a2-314b-4e2e-b8f4-65dc23a859eb None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:20.984610 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8598a2-314b-4e2e-b8f4-65dc23a859eb None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:20.984610 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8598a2-314b-4e2e-b8f4-65dc23a859eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:21.086705 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ee8598a2-314b-4e2e-b8f4-65dc23a859eb None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:21.089136 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ee8598a2-314b-4e2e-b8f4-65dc23a859eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:21.174865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:21.225212 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 660/1324] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:51:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:21.547301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539516d7-21d0-42ff-84ad-3d937c06f30f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:21.549156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-539516d7-21d0-42ff-84ad-3d937c06f30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:21.687624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539516d7-21d0-42ff-84ad-3d937c06f30f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:21.704253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:21.773570 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727ca04-0864-4df2-966d-58a10d246762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:21.774090 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727ca04-0864-4df2-966d-58a10d246762 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:21.774362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727ca04-0864-4df2-966d-58a10d246762 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:21.774628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727ca04-0864-4df2-966d-58a10d246762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:21.776272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d727ca04-0864-4df2-966d-58a10d246762 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:21.912416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727ca04-0864-4df2-966d-58a10d246762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:21.915017 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727ca04-0864-4df2-966d-58a10d246762 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:21.938916 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 665/1325] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:24.423431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d815762-efa3-4efd-bd0d-3e8fedf1170c None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:24.424065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d815762-efa3-4efd-bd0d-3e8fedf1170c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:24.424214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d815762-efa3-4efd-bd0d-3e8fedf1170c None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:24.426400 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 661/1326] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:51:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:51:24.432692 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ce2b9-42fd-4e5c-bd75-a5651811ddf5 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:24.433531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ce2b9-42fd-4e5c-bd75-a5651811ddf5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:24.433814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ce2b9-42fd-4e5c-bd75-a5651811ddf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:24.515960 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:24.539280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-504ce2b9-42fd-4e5c-bd75-a5651811ddf5 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:24.541366 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-504ce2b9-42fd-4e5c-bd75-a5651811ddf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:24.692301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 666/1327] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:51:25.066269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d815762-efa3-4efd-bd0d-3e8fedf1170c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:25.067905 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0d815762-efa3-4efd-bd0d-3e8fedf1170c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:25.198883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d815762-efa3-4efd-bd0d-3e8fedf1170c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:25.292428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c492290-ae07-452c-971e-da5b00ee60fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:25.293520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c492290-ae07-452c-971e-da5b00ee60fa service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:25.297360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c492290-ae07-452c-971e-da5b00ee60fa service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:25.297360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c492290-ae07-452c-971e-da5b00ee60fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:25.297360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2c492290-ae07-452c-971e-da5b00ee60fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:25.328126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:25.447646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c492290-ae07-452c-971e-da5b00ee60fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:25.451474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c492290-ae07-452c-971e-da5b00ee60fa service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:25.486489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 662/1328] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:28.195103 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3ab41-6ef1-4656-884e-8f15f96593e5 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:28.195660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3ab41-6ef1-4656-884e-8f15f96593e5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:28.195660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3ab41-6ef1-4656-884e-8f15f96593e5 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:28.197437 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 667/1329] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:51:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:28.203181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529711b-12f8-4756-8757-0fd7f9e306d0 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:28.203648 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529711b-12f8-4756-8757-0fd7f9e306d0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:28.203770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529711b-12f8-4756-8757-0fd7f9e306d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:28.273429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-2529711b-12f8-4756-8757-0fd7f9e306d0 None None] MFA Rules not processed for user `9f12e9e4aad240d09cdbf43f68f765de`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:28.276002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2529711b-12f8-4756-8757-0fd7f9e306d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:28.350554 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:28.387566 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 663/1330] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:51:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:28.695898 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3ab41-6ef1-4656-884e-8f15f96593e5 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:28.698361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-53c3ab41-6ef1-4656-884e-8f15f96593e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:28.873185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3ab41-6ef1-4656-884e-8f15f96593e5 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:28.900078 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:28.982067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac02340-917b-457a-a61a-4f42b27fc72c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:28.982714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac02340-917b-457a-a61a-4f42b27fc72c service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:28.983023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac02340-917b-457a-a61a-4f42b27fc72c service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:28.983299 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac02340-917b-457a-a61a-4f42b27fc72c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:28.984950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fac02340-917b-457a-a61a-4f42b27fc72c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:29.130028 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac02340-917b-457a-a61a-4f42b27fc72c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:29.131649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac02340-917b-457a-a61a-4f42b27fc72c service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:29.152027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 668/1331] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:31.976537 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736b73-7017-484e-9e9b-2fdd0f71c07c None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:31.977234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736b73-7017-484e-9e9b-2fdd0f71c07c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:31.977426 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736b73-7017-484e-9e9b-2fdd0f71c07c None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:31.979876 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 664/1332] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:51:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:31.985402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f51f9-f170-4ef2-9bcb-a2b5eb4cadc9 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:31.985824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f51f9-f170-4ef2-9bcb-a2b5eb4cadc9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:31.985953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f51f9-f170-4ef2-9bcb-a2b5eb4cadc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:32.055009 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:32.058022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ca6f51f9-f170-4ef2-9bcb-a2b5eb4cadc9 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:32.059291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ca6f51f9-f170-4ef2-9bcb-a2b5eb4cadc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:32.165182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 669/1333] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:32.452184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8736b73-7017-484e-9e9b-2fdd0f71c07c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:32.454037 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d8736b73-7017-484e-9e9b-2fdd0f71c07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:32.579693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8736b73-7017-484e-9e9b-2fdd0f71c07c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:32.671717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3a972-7100-4dae-9dcd-29b7b4d2f69f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:32.672317 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3a972-7100-4dae-9dcd-29b7b4d2f69f service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:32.672594 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3a972-7100-4dae-9dcd-29b7b4d2f69f service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:32.672893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3a972-7100-4dae-9dcd-29b7b4d2f69f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:32.674524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5ac3a972-7100-4dae-9dcd-29b7b4d2f69f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:32.702930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:32.797908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3a972-7100-4dae-9dcd-29b7b4d2f69f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:32.799396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3a972-7100-4dae-9dcd-29b7b4d2f69f service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:32.821137 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 665/1334] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:35.058266 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4b400-7433-4956-ab22-fe945251bdf1 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:35.059153 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4b400-7433-4956-ab22-fe945251bdf1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:35.059153 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4b400-7433-4956-ab22-fe945251bdf1 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:35.060795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 670/1335] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:51:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:35.066776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6f5d2-0207-4976-8c2f-b2073f982ec4 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:35.067319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6f5d2-0207-4976-8c2f-b2073f982ec4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:35.067434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6f5d2-0207-4976-8c2f-b2073f982ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:35.138741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4fe6f5d2-0207-4976-8c2f-b2073f982ec4 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:35.139988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4fe6f5d2-0207-4976-8c2f-b2073f982ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:35.225736 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:35.249694 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 666/1336] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:35.540415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4b400-7433-4956-ab22-fe945251bdf1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:35.542001 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5ce4b400-7433-4956-ab22-fe945251bdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:35.666369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4b400-7433-4956-ab22-fe945251bdf1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:35.696701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:35.744820 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ed3d5-b17e-414d-b327-96131c1e7f9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:35.745345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ed3d5-b17e-414d-b327-96131c1e7f9f service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:35.745626 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ed3d5-b17e-414d-b327-96131c1e7f9f service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:35.746086 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ed3d5-b17e-414d-b327-96131c1e7f9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:35.747610 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ae8ed3d5-b17e-414d-b327-96131c1e7f9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:35.877793 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ed3d5-b17e-414d-b327-96131c1e7f9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:35.879159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ed3d5-b17e-414d-b327-96131c1e7f9f service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:35.898954 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 671/1337] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:38.882670 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da082b8-650c-4067-83f5-426f83ae0498 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:38.883480 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da082b8-650c-4067-83f5-426f83ae0498 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:38.883480 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da082b8-650c-4067-83f5-426f83ae0498 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:38.884624 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 667/1338] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:38.890298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400e477-12cc-42a8-965a-2acd9e792bc6 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:38.890809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400e477-12cc-42a8-965a-2acd9e792bc6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:38.890932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1400e477-12cc-42a8-965a-2acd9e792bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:38.963877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1400e477-12cc-42a8-965a-2acd9e792bc6 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:38.965107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1400e477-12cc-42a8-965a-2acd9e792bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:38.974098 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:39.075045 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 672/1339] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:51:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:51:39.368414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da082b8-650c-4067-83f5-426f83ae0498 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:39.370015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8da082b8-650c-4067-83f5-426f83ae0498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:39.488730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da082b8-650c-4067-83f5-426f83ae0498 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:39.573625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e93f63-d3c7-4b8b-b7d6-a83fb9d26dda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:39.574213 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e93f63-d3c7-4b8b-b7d6-a83fb9d26dda service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:39.574488 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e93f63-d3c7-4b8b-b7d6-a83fb9d26dda service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:39.574764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e93f63-d3c7-4b8b-b7d6-a83fb9d26dda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:39.576547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-47e93f63-d3c7-4b8b-b7d6-a83fb9d26dda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:39.623925 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:39.707919 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e93f63-d3c7-4b8b-b7d6-a83fb9d26dda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:39.709356 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e93f63-d3c7-4b8b-b7d6-a83fb9d26dda service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:39.730134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 668/1340] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:44.619456 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a976129-ada5-426f-8d0f-5c55547ae233 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:44.620801 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a976129-ada5-426f-8d0f-5c55547ae233 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:44.620801 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a976129-ada5-426f-8d0f-5c55547ae233 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:44.621808 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 673/1341] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:44.627471 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb1b9d-edb3-464a-9e01-a26543e2d479 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:44.627958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb1b9d-edb3-464a-9e01-a26543e2d479 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:44.627958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb1b9d-edb3-464a-9e01-a26543e2d479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:44.716368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-29cb1b9d-edb3-464a-9e01-a26543e2d479 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:44.717596 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-29cb1b9d-edb3-464a-9e01-a26543e2d479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:44.824585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:44.841230 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 669/1342] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:51:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:45.140452 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a976129-ada5-426f-8d0f-5c55547ae233 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:45.142005 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0a976129-ada5-426f-8d0f-5c55547ae233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:45.265775 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a976129-ada5-426f-8d0f-5c55547ae233 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:45.312856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:45.370517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f16e0-270e-4171-8c3f-3f983718b9a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:45.372357 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f16e0-270e-4171-8c3f-3f983718b9a5 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:45.372885 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f16e0-270e-4171-8c3f-3f983718b9a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:45.373251 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f16e0-270e-4171-8c3f-3f983718b9a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:45.378002 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-078f16e0-270e-4171-8c3f-3f983718b9a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:45.516771 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f16e0-270e-4171-8c3f-3f983718b9a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:45.518232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f16e0-270e-4171-8c3f-3f983718b9a5 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:45.538694 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 674/1343] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:49.032747 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c98cd-a9a4-4dff-a76e-9d0ed5c7ce65 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:49.033205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c98cd-a9a4-4dff-a76e-9d0ed5c7ce65 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:49.033313 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c98cd-a9a4-4dff-a76e-9d0ed5c7ce65 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:49.035157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 670/1344] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:49.041376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73454ac0-9861-48fa-9834-4b8a96aaa8c3 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:49.042012 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73454ac0-9861-48fa-9834-4b8a96aaa8c3 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:49.042012 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73454ac0-9861-48fa-9834-4b8a96aaa8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:49.127087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-73454ac0-9861-48fa-9834-4b8a96aaa8c3 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:49.128329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-73454ac0-9861-48fa-9834-4b8a96aaa8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:49.149002 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:49.235719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 675/1345] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:51:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:49.523203 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c98cd-a9a4-4dff-a76e-9d0ed5c7ce65 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:49.525125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e52c98cd-a9a4-4dff-a76e-9d0ed5c7ce65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:49.647727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c98cd-a9a4-4dff-a76e-9d0ed5c7ce65 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:49.732724 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52095e10-0a18-4859-861c-a6e1623a57a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:49.733125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52095e10-0a18-4859-861c-a6e1623a57a8 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:49.733401 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52095e10-0a18-4859-861c-a6e1623a57a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:49.733668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52095e10-0a18-4859-861c-a6e1623a57a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:49.735307 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-52095e10-0a18-4859-861c-a6e1623a57a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:49.783376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:49.857309 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52095e10-0a18-4859-861c-a6e1623a57a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:49.858773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52095e10-0a18-4859-861c-a6e1623a57a8 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:49.879323 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 671/1346] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:54.483220 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e311fb3-1f48-4436-bf92-96b0b2899850 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:54.483683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e311fb3-1f48-4436-bf92-96b0b2899850 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:54.483776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e311fb3-1f48-4436-bf92-96b0b2899850 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:54.485132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 676/1347] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:51:54.490874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c966f-14af-4fde-b38c-95c78a5a9327 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:54.491433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c966f-14af-4fde-b38c-95c78a5a9327 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:54.491433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c966f-14af-4fde-b38c-95c78a5a9327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:54.575588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e76c966f-14af-4fde-b38c-95c78a5a9327 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:51:54.577589 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e76c966f-14af-4fde-b38c-95c78a5a9327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:54.712855 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:54.718982 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 672/1348] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:51:55.029155 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e311fb3-1f48-4436-bf92-96b0b2899850 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:51:55.030735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3e311fb3-1f48-4436-bf92-96b0b2899850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:55.139662 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e311fb3-1f48-4436-bf92-96b0b2899850 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:51:55.196272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:55.223737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc91da-8099-425f-9271-d99375b2c8ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:51:55.224345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc91da-8099-425f-9271-d99375b2c8ca service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:51:55.224619 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc91da-8099-425f-9271-d99375b2c8ca service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:51:55.224890 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc91da-8099-425f-9271-d99375b2c8ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:51:55.227049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-29cc91da-8099-425f-9271-d99375b2c8ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:51:55.352443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc91da-8099-425f-9271-d99375b2c8ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:51:55.353889 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc91da-8099-425f-9271-d99375b2c8ca service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:51:55.373576 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 677/1349] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:51:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:00.604900 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e42311-18b1-4988-b6db-e91fe0bdca5e None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:00.605585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e42311-18b1-4988-b6db-e91fe0bdca5e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:00.605585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e42311-18b1-4988-b6db-e91fe0bdca5e None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:00.607437 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 673/1350] 10.222.0.91 () {54 vars in 921 bytes} [Thu Sep 8 13:52:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:00.615458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521ecc2-7f17-4071-81eb-8d7363f23247 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:00.615935 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521ecc2-7f17-4071-81eb-8d7363f23247 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:00.616016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521ecc2-7f17-4071-81eb-8d7363f23247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:00.764597 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f521ecc2-7f17-4071-81eb-8d7363f23247 None None] MFA Rules not processed for user `f9506139d5744fe69153e7e30a05b436`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:00.766810 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f521ecc2-7f17-4071-81eb-8d7363f23247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:00.796494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:01.005718 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 678/1351] 10.222.0.91 () {58 vars in 1018 bytes} [Thu Sep 8 13:52:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:05.969649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9ac5c-315c-4aa2-b5d8-d96054f39a7a None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:05.971087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9ac5c-315c-4aa2-b5d8-d96054f39a7a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:05.971778 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9ac5c-315c-4aa2-b5d8-d96054f39a7a None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:05.975081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 674/1352] 10.222.0.91 () {54 vars in 921 bytes} [Thu Sep 8 13:52:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:05.983086 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f61dc-3725-40c5-9340-5bf6dd9f5eee None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:05.985162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f61dc-3725-40c5-9340-5bf6dd9f5eee None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:05.985162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f61dc-3725-40c5-9340-5bf6dd9f5eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:06.086973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e57f61dc-3725-40c5-9340-5bf6dd9f5eee None None] MFA Rules not processed for user `f9506139d5744fe69153e7e30a05b436`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:06.088143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e57f61dc-3725-40c5-9340-5bf6dd9f5eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:06.185686 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:06.252788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 679/1353] 10.222.0.91 () {58 vars in 1018 bytes} [Thu Sep 8 13:52:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:07.820434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7506a9-ca60-4a31-a835-072acda72939 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:07.820949 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7506a9-ca60-4a31-a835-072acda72939 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:07.820949 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7506a9-ca60-4a31-a835-072acda72939 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:07.822212 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 675/1354] 10.222.0.91 () {54 vars in 921 bytes} [Thu Sep 8 13:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:07.827931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6809d-c078-4812-987c-9e76500d7e68 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:07.828654 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6809d-c078-4812-987c-9e76500d7e68 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:07.828654 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6809d-c078-4812-987c-9e76500d7e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:07.953965 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-0ed6809d-c078-4812-987c-9e76500d7e68 None None] MFA Rules not processed for user `f9506139d5744fe69153e7e30a05b436`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:07.955722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0ed6809d-c078-4812-987c-9e76500d7e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:08.106915 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 680/1355] 10.222.0.91 () {58 vars in 1018 bytes} [Thu Sep 8 13:52:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:11.765499 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b57ee-ebf9-4af8-9808-942fa9378dd3 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:11.766011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b57ee-ebf9-4af8-9808-942fa9378dd3 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:11.766011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b57ee-ebf9-4af8-9808-942fa9378dd3 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:11.767564 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 676/1356] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:11.774024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ed01f-4e57-471d-a439-af89914c27e8 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:11.776069 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ed01f-4e57-471d-a439-af89914c27e8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:11.776069 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ed01f-4e57-471d-a439-af89914c27e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:11.818254 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:11.873816 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-fd2ed01f-4e57-471d-a439-af89914c27e8 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:11.875054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fd2ed01f-4e57-471d-a439-af89914c27e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:12.031870 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 681/1357] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:52:12.372260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6719b56-ac09-4aa9-a814-b313986e876f None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:12.372763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6719b56-ac09-4aa9-a814-b313986e876f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:12.373537 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6719b56-ac09-4aa9-a814-b313986e876f None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:12.376549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 677/1358] 10.222.0.91 () {54 vars in 962 bytes} [Thu Sep 8 13:52:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:12.384731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c36743-787c-42ef-ad79-e75da6a0c813 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:12.385372 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c36743-787c-42ef-ad79-e75da6a0c813 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:12.385804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c36743-787c-42ef-ad79-e75da6a0c813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:12.510013 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-29c36743-787c-42ef-ad79-e75da6a0c813 None None] MFA Rules not processed for user `38127f41852d412ea6ad270f1618fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:12.511279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-29c36743-787c-42ef-ad79-e75da6a0c813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:12.578720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:12.655064 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 682/1359] 10.222.0.91 () {58 vars in 1059 bytes} [Thu Sep 8 13:52:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:52:12.663662 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6719b56-ac09-4aa9-a814-b313986e876f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:12.665180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b6719b56-ac09-4aa9-a814-b313986e876f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:12.836420 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6719b56-ac09-4aa9-a814-b313986e876f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:12.915253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:12.958461 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b71ceb-b5a2-45ec-a6f2-725eb7177601 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:12.959089 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b71ceb-b5a2-45ec-a6f2-725eb7177601 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:12.959420 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b71ceb-b5a2-45ec-a6f2-725eb7177601 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:12.959700 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b71ceb-b5a2-45ec-a6f2-725eb7177601 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:12.961267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e9b71ceb-b5a2-45ec-a6f2-725eb7177601 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:13.059197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b71ceb-b5a2-45ec-a6f2-725eb7177601 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:13.060799 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b71ceb-b5a2-45ec-a6f2-725eb7177601 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:13.079930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 678/1360] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:52:15.630883 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c676e0-1446-4a69-a9e9-decf6e59bc40 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:15.632341 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c676e0-1446-4a69-a9e9-decf6e59bc40 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:15.632341 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c676e0-1446-4a69-a9e9-decf6e59bc40 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:15.633642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 683/1361] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:15.639091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5193fc-573a-406e-817a-03eb6743496a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:15.639835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5193fc-573a-406e-817a-03eb6743496a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:15.639835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5193fc-573a-406e-817a-03eb6743496a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:15.735831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1d5193fc-573a-406e-817a-03eb6743496a None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:15.737587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1d5193fc-573a-406e-817a-03eb6743496a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:15.782353 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:15.849333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 679/1362] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:52:16.144263 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c676e0-1446-4a69-a9e9-decf6e59bc40 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:16.147872 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-87c676e0-1446-4a69-a9e9-decf6e59bc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:16.337093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c676e0-1446-4a69-a9e9-decf6e59bc40 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:16.348695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:16.562036 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf68dc-58d6-4422-8711-d37f46140e9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:16.562656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf68dc-58d6-4422-8711-d37f46140e9d service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:16.563071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf68dc-58d6-4422-8711-d37f46140e9d service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:16.563349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf68dc-58d6-4422-8711-d37f46140e9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:16.564995 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-cbbf68dc-58d6-4422-8711-d37f46140e9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:16.669171 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf68dc-58d6-4422-8711-d37f46140e9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:16.670988 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf68dc-58d6-4422-8711-d37f46140e9d service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:16.694711 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 684/1363] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:19.555206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce1a6b-f255-428c-8ff8-9af2a6b2612c None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:19.555917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce1a6b-f255-428c-8ff8-9af2a6b2612c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:19.555917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce1a6b-f255-428c-8ff8-9af2a6b2612c None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:19.557256 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 680/1364] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:19.562565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada679a4-5518-4420-926a-3664183d5f3e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:19.563039 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada679a4-5518-4420-926a-3664183d5f3e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:19.563234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada679a4-5518-4420-926a-3664183d5f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:19.581196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:19.645284 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ada679a4-5518-4420-926a-3664183d5f3e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:19.647173 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ada679a4-5518-4420-926a-3664183d5f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:19.790586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 685/1365] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:20.106937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0400a-0e73-495f-8fd5-e73516620d53 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.107859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0400a-0e73-495f-8fd5-e73516620d53 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.108705 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0400a-0e73-495f-8fd5-e73516620d53 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:20.110975 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 681/1366] 10.222.0.91 () {54 vars in 962 bytes} [Thu Sep 8 13:52:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:20.117024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028dbea-ef3f-4bb4-b3af-9e9ee8c358f4 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.118549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028dbea-ef3f-4bb4-b3af-9e9ee8c358f4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.118549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028dbea-ef3f-4bb4-b3af-9e9ee8c358f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:20.222699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a028dbea-ef3f-4bb4-b3af-9e9ee8c358f4 None None] MFA Rules not processed for user `38127f41852d412ea6ad270f1618fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:20.224349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a028dbea-ef3f-4bb4-b3af-9e9ee8c358f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:20.275966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:20.415873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 686/1367] 10.222.0.91 () {58 vars in 1059 bytes} [Thu Sep 8 13:52:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:20.426576 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0400a-0e73-495f-8fd5-e73516620d53 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:20.428647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fde0400a-0e73-495f-8fd5-e73516620d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:20.670963 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0400a-0e73-495f-8fd5-e73516620d53 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:20.700231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:20.819880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47841f7c-457f-4c16-96fd-23d9b3589a02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:20.820501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47841f7c-457f-4c16-96fd-23d9b3589a02 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.820776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47841f7c-457f-4c16-96fd-23d9b3589a02 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.821047 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47841f7c-457f-4c16-96fd-23d9b3589a02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:20.822748 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-47841f7c-457f-4c16-96fd-23d9b3589a02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:20.841282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d8043-3ca8-4069-9e3a-c3514868c0f8 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.841871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d8043-3ca8-4069-9e3a-c3514868c0f8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.841871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d8043-3ca8-4069-9e3a-c3514868c0f8 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:20.843573 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 687/1368] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:52:20.849365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d4f4e-af21-4867-9204-b39abf9c7aee None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.849720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d4f4e-af21-4867-9204-b39abf9c7aee None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.849972 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d4f4e-af21-4867-9204-b39abf9c7aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:20.931122 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-243d4f4e-af21-4867-9204-b39abf9c7aee None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:20.932282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-243d4f4e-af21-4867-9204-b39abf9c7aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:20.961098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47841f7c-457f-4c16-96fd-23d9b3589a02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:20.964113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47841f7c-457f-4c16-96fd-23d9b3589a02 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:20.997344 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 682/1369] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:52:21.071656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:21.079705 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 688/1370] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:52:21.403351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc771fd-8589-49c4-9a5b-4847b17c8d93 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.403642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc771fd-8589-49c4-9a5b-4847b17c8d93 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.403923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc771fd-8589-49c4-9a5b-4847b17c8d93 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.405544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 683/1371] 10.222.0.91 () {54 vars in 964 bytes} [Thu Sep 8 13:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:21.411197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3308e65-dc36-4430-891a-1b3a5ea41d27 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.411510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3308e65-dc36-4430-891a-1b3a5ea41d27 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.411784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3308e65-dc36-4430-891a-1b3a5ea41d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.558536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c3308e65-dc36-4430-891a-1b3a5ea41d27 None None] MFA Rules not processed for user `ca225b99584e40f4a4065bf10264974d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:21.560337 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c3308e65-dc36-4430-891a-1b3a5ea41d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:21.675160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 689/1372] 10.222.0.91 () {58 vars in 1061 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:21.682512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc771fd-8589-49c4-9a5b-4847b17c8d93 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.684046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bfc771fd-8589-49c4-9a5b-4847b17c8d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:21.715965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:21.797965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc771fd-8589-49c4-9a5b-4847b17c8d93 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.896325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa074b-706c-42b0-9b11-e940ae02bfd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.897142 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa074b-706c-42b0-9b11-e940ae02bfd1 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.897600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa074b-706c-42b0-9b11-e940ae02bfd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.898044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa074b-706c-42b0-9b11-e940ae02bfd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.900340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-42aa074b-706c-42b0-9b11-e940ae02bfd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:22.064251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:22.123113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa074b-706c-42b0-9b11-e940ae02bfd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.127256 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa074b-706c-42b0-9b11-e940ae02bfd1 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.163645 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 684/1373] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:24.084067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954588f-2e46-42be-8335-c12e80051d69 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:24.084608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954588f-2e46-42be-8335-c12e80051d69 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:24.084608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954588f-2e46-42be-8335-c12e80051d69 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:24.085868 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 690/1374] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:24.091446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014a937-35e6-4f7b-8eb2-a649f18ac699 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:24.091886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014a937-35e6-4f7b-8eb2-a649f18ac699 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:24.091982 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014a937-35e6-4f7b-8eb2-a649f18ac699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:24.178024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-3014a937-35e6-4f7b-8eb2-a649f18ac699 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:24.179361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3014a937-35e6-4f7b-8eb2-a649f18ac699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:24.303276 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 685/1375] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:24.604930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954588f-2e46-42be-8335-c12e80051d69 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:24.606510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b954588f-2e46-42be-8335-c12e80051d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:24.714189 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:24.794261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954588f-2e46-42be-8335-c12e80051d69 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:24.934727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f495f-b7a5-4dc3-8b1f-dc10ab186abe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:24.935392 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f495f-b7a5-4dc3-8b1f-dc10ab186abe service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:24.935673 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f495f-b7a5-4dc3-8b1f-dc10ab186abe service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:24.935947 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f495f-b7a5-4dc3-8b1f-dc10ab186abe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:24.937586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e07f495f-b7a5-4dc3-8b1f-dc10ab186abe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:25.003987 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:25.064276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f495f-b7a5-4dc3-8b1f-dc10ab186abe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:25.065906 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f495f-b7a5-4dc3-8b1f-dc10ab186abe service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:25.089737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 691/1376] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:27.700538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1d123-2331-4ad1-8b56-8e2f876ee4f5 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:27.701079 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1d123-2331-4ad1-8b56-8e2f876ee4f5 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:27.701388 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1d123-2331-4ad1-8b56-8e2f876ee4f5 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:27.703569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 686/1377] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:52:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:52:27.711564 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061603be-27d3-4244-8d8a-19626529d2bc None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:27.712231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061603be-27d3-4244-8d8a-19626529d2bc None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:27.712231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061603be-27d3-4244-8d8a-19626529d2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:27.808216 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-061603be-27d3-4244-8d8a-19626529d2bc None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:27.809459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-061603be-27d3-4244-8d8a-19626529d2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:27.951641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 692/1378] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:52:28.286125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1d123-2331-4ad1-8b56-8e2f876ee4f5 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:28.288994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a9d1d123-2331-4ad1-8b56-8e2f876ee4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:28.341425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:28.513278 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1d123-2331-4ad1-8b56-8e2f876ee4f5 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:28.718333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:28.727353 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a4556-47e2-427d-86e6-62d55cfa79a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:28.728351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4556-47e2-427d-86e6-62d55cfa79a5 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:28.728785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4556-47e2-427d-86e6-62d55cfa79a5 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:28.733801 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4556-47e2-427d-86e6-62d55cfa79a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:28.733801 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ee9a4556-47e2-427d-86e6-62d55cfa79a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:28.875847 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a4556-47e2-427d-86e6-62d55cfa79a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:28.877661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a4556-47e2-427d-86e6-62d55cfa79a5 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:28.906507 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 687/1379] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:31.392113 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab998e9-988f-4a8b-bb1f-6667ea330852 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:31.392715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab998e9-988f-4a8b-bb1f-6667ea330852 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:31.392715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab998e9-988f-4a8b-bb1f-6667ea330852 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:31.394009 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 693/1380] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:31.400341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5686d-7695-497b-8dc1-dff1c1f0bb62 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:31.400968 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5686d-7695-497b-8dc1-dff1c1f0bb62 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:31.401135 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5686d-7695-497b-8dc1-dff1c1f0bb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:31.470403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-dbc5686d-7695-497b-8dc1-dff1c1f0bb62 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:31.471627 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-dbc5686d-7695-497b-8dc1-dff1c1f0bb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:31.553449 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:31.593799 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 688/1381] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:31.995358 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab998e9-988f-4a8b-bb1f-6667ea330852 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:31.996954 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-cab998e9-988f-4a8b-bb1f-6667ea330852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:32.041413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:32.188515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab998e9-988f-4a8b-bb1f-6667ea330852 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:32.316612 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:32.334834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875ab6a-eff1-44c4-a158-b4e62f28278c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:32.335426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875ab6a-eff1-44c4-a158-b4e62f28278c service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:32.335720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875ab6a-eff1-44c4-a158-b4e62f28278c service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:32.336006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875ab6a-eff1-44c4-a158-b4e62f28278c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:32.337661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b875ab6a-eff1-44c4-a158-b4e62f28278c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:32.446194 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875ab6a-eff1-44c4-a158-b4e62f28278c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:32.447868 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875ab6a-eff1-44c4-a158-b4e62f28278c service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:32.469057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 694/1382] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:35.621632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c03f4-c9e9-4116-b9f7-81fbbcda92bc None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:35.622513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c03f4-c9e9-4116-b9f7-81fbbcda92bc None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:35.622513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c03f4-c9e9-4116-b9f7-81fbbcda92bc None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:35.624187 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 689/1383] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:52:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:52:35.633203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df793886-872a-4238-86ba-662b14df7dc9 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:35.633857 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df793886-872a-4238-86ba-662b14df7dc9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:35.634166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df793886-872a-4238-86ba-662b14df7dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:35.708914 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-df793886-872a-4238-86ba-662b14df7dc9 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:35.710179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-df793886-872a-4238-86ba-662b14df7dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:35.769242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:35.815151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 695/1384] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:36.125586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c03f4-c9e9-4116-b9f7-81fbbcda92bc None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:36.127588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5a4c03f4-c9e9-4116-b9f7-81fbbcda92bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:36.332579 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c03f4-c9e9-4116-b9f7-81fbbcda92bc None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:36.394533 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:36.465727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5642bd-d20a-449e-ab23-a56048e57cc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:36.466351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5642bd-d20a-449e-ab23-a56048e57cc3 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:36.466644 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5642bd-d20a-449e-ab23-a56048e57cc3 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:36.466949 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5642bd-d20a-449e-ab23-a56048e57cc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:36.468877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6f5642bd-d20a-449e-ab23-a56048e57cc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:36.572212 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5642bd-d20a-449e-ab23-a56048e57cc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:36.574215 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5642bd-d20a-449e-ab23-a56048e57cc3 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:36.597119 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 690/1385] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:38.808288 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2331f4-cf43-4583-a6f0-c9eb06bdd1ee None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:38.809118 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2331f4-cf43-4583-a6f0-c9eb06bdd1ee None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:38.809118 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2331f4-cf43-4583-a6f0-c9eb06bdd1ee None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:38.810073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 696/1386] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:52:38.816546 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5565abc-99a6-4b4e-99c7-6661bda11578 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:38.817262 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5565abc-99a6-4b4e-99c7-6661bda11578 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:38.818097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5565abc-99a6-4b4e-99c7-6661bda11578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:38.897823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e5565abc-99a6-4b4e-99c7-6661bda11578 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:38.899055 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e5565abc-99a6-4b4e-99c7-6661bda11578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:38.907936 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:39.033250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 691/1387] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:39.237606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609267e-e84a-4c30-a17d-b3d3a0cb874a None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:39.237986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609267e-e84a-4c30-a17d-b3d3a0cb874a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:39.238230 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609267e-e84a-4c30-a17d-b3d3a0cb874a None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:39.239681 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 697/1388] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:39.246062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3426b0-dc5f-4fa4-b316-25c3c93ef78b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:39.246518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3426b0-dc5f-4fa4-b316-25c3c93ef78b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:39.246778 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3426b0-dc5f-4fa4-b316-25c3c93ef78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:39.323105 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4f3426b0-dc5f-4fa4-b316-25c3c93ef78b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:39.324321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4f3426b0-dc5f-4fa4-b316-25c3c93ef78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:39.336093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609267e-e84a-4c30-a17d-b3d3a0cb874a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:39.337654 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e609267e-e84a-4c30-a17d-b3d3a0cb874a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:39.417172 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:39.428058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 692/1389] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:39.500858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609267e-e84a-4c30-a17d-b3d3a0cb874a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:39.549571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:39.623693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf674f4-3fa8-4597-9737-4945b6153e0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:39.624303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf674f4-3fa8-4597-9737-4945b6153e0d service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:39.624564 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf674f4-3fa8-4597-9737-4945b6153e0d service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:39.624825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf674f4-3fa8-4597-9737-4945b6153e0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:39.626856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-eaf674f4-3fa8-4597-9737-4945b6153e0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:39.725783 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf674f4-3fa8-4597-9737-4945b6153e0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:39.727431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf674f4-3fa8-4597-9737-4945b6153e0d service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:39.732730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850b2cb-5432-4c32-8b57-4c3abb3bd08a None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:39.733179 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850b2cb-5432-4c32-8b57-4c3abb3bd08a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:39.733655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850b2cb-5432-4c32-8b57-4c3abb3bd08a None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:39.736360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 693/1390] 10.222.0.91 () {54 vars in 961 bytes} [Thu Sep 8 13:52:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:39.745086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c7ee8-a089-4695-927f-b9f85f546232 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:39.745462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c7ee8-a089-4695-927f-b9f85f546232 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:39.745654 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c7ee8-a089-4695-927f-b9f85f546232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:39.746688 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 698/1391] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:39.851797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-586c7ee8-a089-4695-927f-b9f85f546232 None None] MFA Rules not processed for user `01b060c70e1b440186eeac37d9c34567`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:39.853349 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-586c7ee8-a089-4695-927f-b9f85f546232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:39.937930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 694/1392] 10.222.0.91 () {58 vars in 1058 bytes} [Thu Sep 8 13:52:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:39.947811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf674f4-3fa8-4597-9737-4945b6153e0d service nova] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:39.949026 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-eaf674f4-3fa8-4597-9737-4945b6153e0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:40.018739 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:40.052814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf674f4-3fa8-4597-9737-4945b6153e0d service nova] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:40.145472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4afbf-a1a6-4020-b5e5-3ea9d05237e9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b060c70e1b440186eeac37d9c34567', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:40.146091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4afbf-a1a6-4020-b5e5-3ea9d05237e9 service glance] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.146371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4afbf-a1a6-4020-b5e5-3ea9d05237e9 service glance] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.146640 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4afbf-a1a6-4020-b5e5-3ea9d05237e9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.148353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f8f4afbf-a1a6-4020-b5e5-3ea9d05237e9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:40.284685 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4afbf-a1a6-4020-b5e5-3ea9d05237e9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:40.287038 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4afbf-a1a6-4020-b5e5-3ea9d05237e9 service glance] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:40.314191 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 699/1393] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:41.244819 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbb504-39b2-459e-812c-f8c8f2a57bb7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:41.245314 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbb504-39b2-459e-812c-f8c8f2a57bb7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:41.245411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbb504-39b2-459e-812c-f8c8f2a57bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:41.321063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-eccbb504-39b2-459e-812c-f8c8f2a57bb7 None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:41.322267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-eccbb504-39b2-459e-812c-f8c8f2a57bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:41.360260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:41.414192 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 695/1394] 10.222.0.91 () {58 vars in 1009 bytes} [Thu Sep 8 13:52:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:41.464004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b390661-e454-468d-8ef8-e1ab21f64556 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:41.464507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b390661-e454-468d-8ef8-e1ab21f64556 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:41.464614 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b390661-e454-468d-8ef8-e1ab21f64556 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:41.465895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 700/1395] 10.222.0.91 () {54 vars in 943 bytes} [Thu Sep 8 13:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:41.472192 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece6f8f-8ffa-4a91-8353-21d5acd7a739 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:41.472476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece6f8f-8ffa-4a91-8353-21d5acd7a739 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:41.472755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece6f8f-8ffa-4a91-8353-21d5acd7a739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:41.552509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-aece6f8f-8ffa-4a91-8353-21d5acd7a739 None None] MFA Rules not processed for user `b3b5ae758fe847adaae0edaa0be095c9`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:41.553774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-aece6f8f-8ffa-4a91-8353-21d5acd7a739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:41.687610 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 696/1396] 10.222.0.91 () {58 vars in 1040 bytes} [Thu Sep 8 13:52:41 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:52:41.697546 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b390661-e454-468d-8ef8-e1ab21f64556 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:41.704323 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7b390661-e454-468d-8ef8-e1ab21f64556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:41.709998 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:41.854694 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b390661-e454-468d-8ef8-e1ab21f64556 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:41.965859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:41.972495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8d174-5b25-43bb-9c5e-1e9dc9e4b955 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:41.973498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8d174-5b25-43bb-9c5e-1e9dc9e4b955 service swift] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:41.973783 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8d174-5b25-43bb-9c5e-1e9dc9e4b955 service swift] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:41.974084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8d174-5b25-43bb-9c5e-1e9dc9e4b955 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:41.975800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-79e8d174-5b25-43bb-9c5e-1e9dc9e4b955 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:42.069956 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8d174-5b25-43bb-9c5e-1e9dc9e4b955 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:42.072101 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e8d174-5b25-43bb-9c5e-1e9dc9e4b955 service swift] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:42.080442 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 701/1397] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 13:52:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:52:42.150655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2d356-884f-47e6-92a2-912ae6301b5a None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:42.150972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2d356-884f-47e6-92a2-912ae6301b5a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:42.151247 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2d356-884f-47e6-92a2-912ae6301b5a None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:42.152602 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 697/1398] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:42.158457 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6c9d0-5595-4883-9b07-7bafa7fdab9a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:42.158742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6c9d0-5595-4883-9b07-7bafa7fdab9a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:42.159052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c6c9d0-5595-4883-9b07-7bafa7fdab9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:42.227167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e9c6c9d0-5595-4883-9b07-7bafa7fdab9a None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:42.228304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e9c6c9d0-5595-4883-9b07-7bafa7fdab9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:42.316840 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:42.336738 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 702/1399] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:42.681854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2d356-884f-47e6-92a2-912ae6301b5a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:42.683427 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-aad2d356-884f-47e6-92a2-912ae6301b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:42.697170 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:42.897369 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2d356-884f-47e6-92a2-912ae6301b5a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:43.016547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:43.071814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f28b8-2ad6-4589-9abe-9139bb6f7f0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:43.072524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f28b8-2ad6-4589-9abe-9139bb6f7f0a service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:43.072863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f28b8-2ad6-4589-9abe-9139bb6f7f0a service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:43.073232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f28b8-2ad6-4589-9abe-9139bb6f7f0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:43.074926 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-de1f28b8-2ad6-4589-9abe-9139bb6f7f0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:43.200102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f28b8-2ad6-4589-9abe-9139bb6f7f0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:43.202137 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f28b8-2ad6-4589-9abe-9139bb6f7f0a service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:43.233064 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 698/1400] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:45.741632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65421e-9b65-4588-8496-127d0b41813b None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:45.743089 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65421e-9b65-4588-8496-127d0b41813b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:45.743089 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65421e-9b65-4588-8496-127d0b41813b None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:45.743774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 703/1401] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:45.749337 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e5162-c103-4bf0-ae80-448d3bb8c5a7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:45.749985 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e5162-c103-4bf0-ae80-448d3bb8c5a7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:45.749985 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e5162-c103-4bf0-ae80-448d3bb8c5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:45.844004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c29e5162-c103-4bf0-ae80-448d3bb8c5a7 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:45.845248 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c29e5162-c103-4bf0-ae80-448d3bb8c5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:45.896835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:45.962939 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 699/1402] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:46.262433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65421e-9b65-4588-8496-127d0b41813b None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:46.267889 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5c65421e-9b65-4588-8496-127d0b41813b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:46.481341 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65421e-9b65-4588-8496-127d0b41813b None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:46.580808 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:46.660012 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2473122c-a891-46ee-9b15-9d518f4e9302 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:46.660629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473122c-a891-46ee-9b15-9d518f4e9302 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:46.661568 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473122c-a891-46ee-9b15-9d518f4e9302 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:46.661878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473122c-a891-46ee-9b15-9d518f4e9302 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:46.663611 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2473122c-a891-46ee-9b15-9d518f4e9302 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:46.808140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2473122c-a891-46ee-9b15-9d518f4e9302 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:46.811189 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2473122c-a891-46ee-9b15-9d518f4e9302 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:46.833481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 704/1403] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:49.243893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9e4d8-c8ac-47d1-b20f-2ab66cc60c3d None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:49.244473 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9e4d8-c8ac-47d1-b20f-2ab66cc60c3d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:49.244837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9e4d8-c8ac-47d1-b20f-2ab66cc60c3d None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:49.246483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 700/1404] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:52:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:49.256084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c7db1-ce68-4574-9ae8-f55c034701c3 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:49.256611 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c7db1-ce68-4574-9ae8-f55c034701c3 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:49.256611 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c7db1-ce68-4574-9ae8-f55c034701c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:49.336413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ec2c7db1-ce68-4574-9ae8-f55c034701c3 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:49.337594 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ec2c7db1-ce68-4574-9ae8-f55c034701c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:49.374785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:49.572058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 705/1405] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:49.874782 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9e4d8-c8ac-47d1-b20f-2ab66cc60c3d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:49.876366 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-80c9e4d8-c8ac-47d1-b20f-2ab66cc60c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:50.110392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9e4d8-c8ac-47d1-b20f-2ab66cc60c3d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:50.165569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:50.284171 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bc473-f039-474e-8553-f609109c03d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:50.285390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bc473-f039-474e-8553-f609109c03d6 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:50.285758 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bc473-f039-474e-8553-f609109c03d6 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:50.286065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bc473-f039-474e-8553-f609109c03d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:50.288321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-606bc473-f039-474e-8553-f609109c03d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:50.440907 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606bc473-f039-474e-8553-f609109c03d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:50.442798 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606bc473-f039-474e-8553-f609109c03d6 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:50.468667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 701/1406] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:53.178210 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54894721-8df0-4756-9de3-4c404b355a00 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:53.178785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54894721-8df0-4756-9de3-4c404b355a00 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:53.178785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54894721-8df0-4756-9de3-4c404b355a00 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:53.180278 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 706/1407] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:53.185987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f71b0-6111-44eb-81f8-66e2e529581a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:53.186455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f71b0-6111-44eb-81f8-66e2e529581a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:53.186546 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f71b0-6111-44eb-81f8-66e2e529581a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:53.238923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:53.273413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-bf8f71b0-6111-44eb-81f8-66e2e529581a None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:53.274638 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bf8f71b0-6111-44eb-81f8-66e2e529581a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:53.400164 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 702/1408] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:53.696525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54894721-8df0-4756-9de3-4c404b355a00 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:53.698949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-54894721-8df0-4756-9de3-4c404b355a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:53.914189 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54894721-8df0-4756-9de3-4c404b355a00 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:53.933805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:54.077615 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a0cd4-e5be-49bb-a609-220f46f9fb16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:54.078255 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a0cd4-e5be-49bb-a609-220f46f9fb16 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:54.078518 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a0cd4-e5be-49bb-a609-220f46f9fb16 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:54.078792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a0cd4-e5be-49bb-a609-220f46f9fb16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:54.080554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-435a0cd4-e5be-49bb-a609-220f46f9fb16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:54.198782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a0cd4-e5be-49bb-a609-220f46f9fb16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:54.201245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a0cd4-e5be-49bb-a609-220f46f9fb16 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:54.224114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 707/1409] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:03.398513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=51 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:04.220721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:06.772042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bb299-568e-46ba-a9ce-ee7fa0d7fe29 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.772760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bb299-568e-46ba-a9ce-ee7fa0d7fe29 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.772760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669bb299-568e-46ba-a9ce-ee7fa0d7fe29 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.773860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 703/1410] 10.222.0.91 () {54 vars in 918 bytes} [Thu Sep 8 13:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:06.779417 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeb3a3-c449-4b21-842b-fc93a851581b None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.779907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeb3a3-c449-4b21-842b-fc93a851581b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.782986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeb3a3-c449-4b21-842b-fc93a851581b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.860771 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ebdeb3a3-c449-4b21-842b-fc93a851581b None None] MFA Rules not processed for user `f9506139d5744fe69153e7e30a05b436`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:06.862004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ebdeb3a3-c449-4b21-842b-fc93a851581b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:06.972271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 708/1411] 10.222.0.91 () {58 vars in 1015 bytes} [Thu Sep 8 13:53:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:06.987860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea074c4-05aa-4a04-9e20-fe6427ec5c1b None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.988147 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea074c4-05aa-4a04-9e20-fe6427ec5c1b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.989039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea074c4-05aa-4a04-9e20-fe6427ec5c1b None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.990499 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 704/1412] 10.222.0.91 () {54 vars in 918 bytes} [Thu Sep 8 13:53:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:06.995501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e1587-e7c4-403a-8825-b19c30fe36d1 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.995791 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e1587-e7c4-403a-8825-b19c30fe36d1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.996069 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e1587-e7c4-403a-8825-b19c30fe36d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.074758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:07.077861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e60e1587-e7c4-403a-8825-b19c30fe36d1 None None] MFA Rules not processed for user `38127f41852d412ea6ad270f1618fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:07.079145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e60e1587-e7c4-403a-8825-b19c30fe36d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:07.194922 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 709/1413] 10.222.0.91 () {58 vars in 1015 bytes} [Thu Sep 8 13:53:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:09.780788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96327d-b9d4-49b1-8ab6-dd11880f1bf2 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:09.781935 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96327d-b9d4-49b1-8ab6-dd11880f1bf2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:09.781935 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96327d-b9d4-49b1-8ab6-dd11880f1bf2 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:09.782875 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:09.789425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d17996-cfe3-4941-8e13-8e20de9a242e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:09.789937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d17996-cfe3-4941-8e13-8e20de9a242e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:09.790131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d17996-cfe3-4941-8e13-8e20de9a242e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:09.864027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b0d17996-cfe3-4941-8e13-8e20de9a242e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:09.865276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b0d17996-cfe3-4941-8e13-8e20de9a242e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:09.927688 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:09.976983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:10.274924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96327d-b9d4-49b1-8ab6-dd11880f1bf2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:10.276483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bf96327d-b9d4-49b1-8ab6-dd11880f1bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:10.439421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96327d-b9d4-49b1-8ab6-dd11880f1bf2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:10.517320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:10.561324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68508c1-990f-4657-b0e0-da2a2c45f191 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:10.561927 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68508c1-990f-4657-b0e0-da2a2c45f191 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:10.562203 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68508c1-990f-4657-b0e0-da2a2c45f191 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:10.562588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68508c1-990f-4657-b0e0-da2a2c45f191 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:10.564148 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e68508c1-990f-4657-b0e0-da2a2c45f191 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:10.658926 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68508c1-990f-4657-b0e0-da2a2c45f191 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:10.661166 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68508c1-990f-4657-b0e0-da2a2c45f191 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:10.680489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 706/1416] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:11.831415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97108709-197c-4ffe-9cd3-7a5cfc5be511 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:11.832426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97108709-197c-4ffe-9cd3-7a5cfc5be511 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:11.832426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97108709-197c-4ffe-9cd3-7a5cfc5be511 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:11.833719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:11.840012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b56e4a-090b-4dcf-b69e-0c928db3d2fa None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:11.840434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b56e4a-090b-4dcf-b69e-0c928db3d2fa None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:11.840553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b56e4a-090b-4dcf-b69e-0c928db3d2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:11.916036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c5b56e4a-090b-4dcf-b69e-0c928db3d2fa None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:11.917244 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c5b56e4a-090b-4dcf-b69e-0c928db3d2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:11.943569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:12.012441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:12.163063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97108709-197c-4ffe-9cd3-7a5cfc5be511 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:12.164537 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-97108709-197c-4ffe-9cd3-7a5cfc5be511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:12.324168 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97108709-197c-4ffe-9cd3-7a5cfc5be511 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:12.366251 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:12.587921 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39853c62-cf96-4065-aaff-f8af3606a5b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:12.588524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39853c62-cf96-4065-aaff-f8af3606a5b2 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:12.588800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39853c62-cf96-4065-aaff-f8af3606a5b2 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:12.589071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39853c62-cf96-4065-aaff-f8af3606a5b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:12.590779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-39853c62-cf96-4065-aaff-f8af3606a5b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:12.680884 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39853c62-cf96-4065-aaff-f8af3606a5b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:12.682468 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39853c62-cf96-4065-aaff-f8af3606a5b2 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:12.699966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 712/1419] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:13.870472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199981f-7416-4faa-afc1-2902c7ff94ed None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:13.871072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199981f-7416-4faa-afc1-2902c7ff94ed None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:13.871072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199981f-7416-4faa-afc1-2902c7ff94ed None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:13.872412 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:13.879076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecfb2a-6fed-4b50-b7df-c5f5f531ca0d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:13.879564 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecfb2a-6fed-4b50-b7df-c5f5f531ca0d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:13.879690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ecfb2a-6fed-4b50-b7df-c5f5f531ca0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:13.935300 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:13.956575 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-07ecfb2a-6fed-4b50-b7df-c5f5f531ca0d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:13.957788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-07ecfb2a-6fed-4b50-b7df-c5f5f531ca0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:14.052454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:14.206143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199981f-7416-4faa-afc1-2902c7ff94ed None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:14.207628 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f199981f-7416-4faa-afc1-2902c7ff94ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:14.351906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199981f-7416-4faa-afc1-2902c7ff94ed None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:14.363943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:14.468130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877421ce-cce5-4b24-87f7-769dc32bb1b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:14.468636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877421ce-cce5-4b24-87f7-769dc32bb1b2 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:14.468911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877421ce-cce5-4b24-87f7-769dc32bb1b2 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:14.469174 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877421ce-cce5-4b24-87f7-769dc32bb1b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:14.470759 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-877421ce-cce5-4b24-87f7-769dc32bb1b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:14.561273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877421ce-cce5-4b24-87f7-769dc32bb1b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:14.562860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877421ce-cce5-4b24-87f7-769dc32bb1b2 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:14.582611 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 709/1422] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:15.716217 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13d4f5-899f-4eb5-b171-a04873b7d16d None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:15.716839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13d4f5-899f-4eb5-b171-a04873b7d16d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:15.717113 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13d4f5-899f-4eb5-b171-a04873b7d16d None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:15.718526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:53:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:15.726479 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1301c-bdb5-4fb0-93d9-1493b2142bf0 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:15.727058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1301c-bdb5-4fb0-93d9-1493b2142bf0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:15.727313 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1301c-bdb5-4fb0-93d9-1493b2142bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:15.751731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:15.817904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e8c1301c-bdb5-4fb0-93d9-1493b2142bf0 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:15.819172 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e8c1301c-bdb5-4fb0-93d9-1493b2142bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:15.924175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:53:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:16.085038 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13d4f5-899f-4eb5-b171-a04873b7d16d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:16.085038 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-dc13d4f5-899f-4eb5-b171-a04873b7d16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:16.204047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:16.241587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13d4f5-899f-4eb5-b171-a04873b7d16d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:16.364731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bae65b-ff0a-4f67-8c68-16619f7d42ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:16.365336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bae65b-ff0a-4f67-8c68-16619f7d42ff service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:16.365609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bae65b-ff0a-4f67-8c68-16619f7d42ff service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:16.365878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bae65b-ff0a-4f67-8c68-16619f7d42ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:16.367703 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-96bae65b-ff0a-4f67-8c68-16619f7d42ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:16.453209 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:16.468031 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bae65b-ff0a-4f67-8c68-16619f7d42ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:16.469641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bae65b-ff0a-4f67-8c68-16619f7d42ff service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:16.493179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 715/1425] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:17.628236 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359fd56-3fcb-4017-a100-8bd807c06390 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:17.628808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359fd56-3fcb-4017-a100-8bd807c06390 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:17.628808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359fd56-3fcb-4017-a100-8bd807c06390 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:17.630123 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:17.636304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc9b7e-887b-4eed-a487-bf0645b0196f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:17.636730 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc9b7e-887b-4eed-a487-bf0645b0196f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:17.636857 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc9b7e-887b-4eed-a487-bf0645b0196f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:17.721074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-4dbc9b7e-887b-4eed-a487-bf0645b0196f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:17.722829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4dbc9b7e-887b-4eed-a487-bf0645b0196f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:17.819504 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:17.970962 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359fd56-3fcb-4017-a100-8bd807c06390 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:17.973074 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c359fd56-3fcb-4017-a100-8bd807c06390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:18.057287 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:18.120776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359fd56-3fcb-4017-a100-8bd807c06390 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:18.241534 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e602b3-cabe-4cb5-9c21-cbcb8b788d45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:18.242120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e602b3-cabe-4cb5-9c21-cbcb8b788d45 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:18.242394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e602b3-cabe-4cb5-9c21-cbcb8b788d45 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:18.242658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e602b3-cabe-4cb5-9c21-cbcb8b788d45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:18.244286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-88e602b3-cabe-4cb5-9c21-cbcb8b788d45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:18.295540 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:18.339043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e602b3-cabe-4cb5-9c21-cbcb8b788d45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:18.340652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e602b3-cabe-4cb5-9c21-cbcb8b788d45 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:18.358656 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 712/1428] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:19.486660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70cd9a-1ce0-4465-b8aa-ff2648a5302b None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:19.487169 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70cd9a-1ce0-4465-b8aa-ff2648a5302b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:19.487277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd70cd9a-1ce0-4465-b8aa-ff2648a5302b None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:19.489075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:19.495474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45a840-59cf-4d80-9beb-7b494af215d2 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:19.495942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45a840-59cf-4d80-9beb-7b494af215d2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:19.496050 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45a840-59cf-4d80-9beb-7b494af215d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:19.573845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4c45a840-59cf-4d80-9beb-7b494af215d2 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:19.575098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4c45a840-59cf-4d80-9beb-7b494af215d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:19.671208 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:19.824670 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70cd9a-1ce0-4465-b8aa-ff2648a5302b None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:19.826125 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-dd70cd9a-1ce0-4465-b8aa-ff2648a5302b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:19.876628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:19.989350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd70cd9a-1ce0-4465-b8aa-ff2648a5302b None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:20.110739 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c9d32-746d-4022-8530-85ea0cf2e5b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:20.111364 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c9d32-746d-4022-8530-85ea0cf2e5b8 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:20.111647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c9d32-746d-4022-8530-85ea0cf2e5b8 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:20.111916 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c9d32-746d-4022-8530-85ea0cf2e5b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:20.114013 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7d5c9d32-746d-4022-8530-85ea0cf2e5b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:20.121088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:20.206350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c9d32-746d-4022-8530-85ea0cf2e5b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:20.207973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c9d32-746d-4022-8530-85ea0cf2e5b8 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:20.225622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 718/1431] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:21.343996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97c39b-ca4b-4d7f-b738-54700aeb55ad None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.344582 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97c39b-ca4b-4d7f-b738-54700aeb55ad None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.344582 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97c39b-ca4b-4d7f-b738-54700aeb55ad None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.345874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:21.352733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbc6e9-8819-4efd-8512-0db32c50b40a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.353198 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbc6e9-8819-4efd-8512-0db32c50b40a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.353294 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbc6e9-8819-4efd-8512-0db32c50b40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.432009 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ddcbc6e9-8819-4efd-8512-0db32c50b40a None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:21.433240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ddcbc6e9-8819-4efd-8512-0db32c50b40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:21.505490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:21.535533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:21.689451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97c39b-ca4b-4d7f-b738-54700aeb55ad None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:21.691650 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ef97c39b-ca4b-4d7f-b738-54700aeb55ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:21.709085 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:21.843850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97c39b-ca4b-4d7f-b738-54700aeb55ad None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:21.937383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:21.964963 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887607d-919c-4749-a7e0-b8adf42ff35d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:21.965486 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887607d-919c-4749-a7e0-b8adf42ff35d service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.965767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887607d-919c-4749-a7e0-b8adf42ff35d service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.966035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887607d-919c-4749-a7e0-b8adf42ff35d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.967710 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c887607d-919c-4749-a7e0-b8adf42ff35d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:22.062954 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887607d-919c-4749-a7e0-b8adf42ff35d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:22.064527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c887607d-919c-4749-a7e0-b8adf42ff35d service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:22.082984 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 715/1434] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:23.220813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d70f16-79a7-4165-bca1-07babe1d84b4 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.221395 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d70f16-79a7-4165-bca1-07babe1d84b4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.221395 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d70f16-79a7-4165-bca1-07babe1d84b4 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.222750 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:23.228981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f30dce-e466-46e5-886a-7d30bbec6858 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.229408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f30dce-e466-46e5-886a-7d30bbec6858 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.229520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f30dce-e466-46e5-886a-7d30bbec6858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.304214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f2f30dce-e466-46e5-886a-7d30bbec6858 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:23.305408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f2f30dce-e466-46e5-886a-7d30bbec6858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:23.347836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:23.400312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:23.552655 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d70f16-79a7-4165-bca1-07babe1d84b4 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:23.554176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b3d70f16-79a7-4165-bca1-07babe1d84b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:23.701158 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d70f16-79a7-4165-bca1-07babe1d84b4 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:23.755789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:23.813768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18b8f9-b2aa-484f-a120-cd97c1a582be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:23.814366 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18b8f9-b2aa-484f-a120-cd97c1a582be service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.814641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18b8f9-b2aa-484f-a120-cd97c1a582be service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.814917 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18b8f9-b2aa-484f-a120-cd97c1a582be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.816573 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2f18b8f9-b2aa-484f-a120-cd97c1a582be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:23.907579 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18b8f9-b2aa-484f-a120-cd97c1a582be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:23.909192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18b8f9-b2aa-484f-a120-cd97c1a582be service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:23.928143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 721/1437] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:25.073548 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16647dd1-5e10-43ae-a1da-0b1ef6f59708 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:25.074093 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16647dd1-5e10-43ae-a1da-0b1ef6f59708 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:25.074093 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16647dd1-5e10-43ae-a1da-0b1ef6f59708 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:25.075456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:25.082868 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e89be-62ee-42b4-8d8b-861d0c1c789e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:25.083452 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e89be-62ee-42b4-8d8b-861d0c1c789e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:25.083789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e89be-62ee-42b4-8d8b-861d0c1c789e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:25.160327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-898e89be-62ee-42b4-8d8b-861d0c1c789e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:25.161682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-898e89be-62ee-42b4-8d8b-861d0c1c789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:25.163730 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:25.257374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:25.406839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16647dd1-5e10-43ae-a1da-0b1ef6f59708 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:25.408372 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-16647dd1-5e10-43ae-a1da-0b1ef6f59708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:25.553809 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16647dd1-5e10-43ae-a1da-0b1ef6f59708 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:25.582424 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:25.669176 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33985d04-27b2-4bdb-ba51-1b33342c1b9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:25.669669 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33985d04-27b2-4bdb-ba51-1b33342c1b9d service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:25.669941 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33985d04-27b2-4bdb-ba51-1b33342c1b9d service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:25.670209 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33985d04-27b2-4bdb-ba51-1b33342c1b9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:25.671860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-33985d04-27b2-4bdb-ba51-1b33342c1b9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:25.768474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33985d04-27b2-4bdb-ba51-1b33342c1b9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:25.770133 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33985d04-27b2-4bdb-ba51-1b33342c1b9d service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:25.788571 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 718/1440] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:27.860731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10ffff-2a4b-4ed3-89de-2512121875c5 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.861270 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10ffff-2a4b-4ed3-89de-2512121875c5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.861270 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10ffff-2a4b-4ed3-89de-2512121875c5 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.862647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:27.868235 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bf1b9-8c8c-44eb-9eb6-3704ad0e840d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.868639 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bf1b9-8c8c-44eb-9eb6-3704ad0e840d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.868807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8bf1b9-8c8c-44eb-9eb6-3704ad0e840d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.911834 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:27.938411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ba8bf1b9-8c8c-44eb-9eb6-3704ad0e840d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:27.940339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ba8bf1b9-8c8c-44eb-9eb6-3704ad0e840d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:28.039010 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:28.325292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10ffff-2a4b-4ed3-89de-2512121875c5 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.326834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-dd10ffff-2a4b-4ed3-89de-2512121875c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:28.430971 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10ffff-2a4b-4ed3-89de-2512121875c5 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.463450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:28.506203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a6df2-cd08-4b4c-ba6c-2662588cf3c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.506800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a6df2-cd08-4b4c-ba6c-2662588cf3c6 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.507121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a6df2-cd08-4b4c-ba6c-2662588cf3c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.507391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a6df2-cd08-4b4c-ba6c-2662588cf3c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.508975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9d8a6df2-cd08-4b4c-ba6c-2662588cf3c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:28.618480 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a6df2-cd08-4b4c-ba6c-2662588cf3c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.619924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a6df2-cd08-4b4c-ba6c-2662588cf3c6 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.637792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 724/1443] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:30.970957 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70165d3-3c7b-45f6-91bf-3f1cc202d896 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:30.971522 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70165d3-3c7b-45f6-91bf-3f1cc202d896 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:30.971522 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70165d3-3c7b-45f6-91bf-3f1cc202d896 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:30.973696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:53:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:30.979141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1383c7-bb3f-480b-9cff-cbba4bdad790 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:30.979575 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1383c7-bb3f-480b-9cff-cbba4bdad790 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:30.979679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1383c7-bb3f-480b-9cff-cbba4bdad790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:31.045046 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-9b1383c7-bb3f-480b-9cff-cbba4bdad790 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:31.046258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9b1383c7-bb3f-480b-9cff-cbba4bdad790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:31.060361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:31.143919 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:53:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:31.428108 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70165d3-3c7b-45f6-91bf-3f1cc202d896 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:31.429651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e70165d3-3c7b-45f6-91bf-3f1cc202d896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:31.534183 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70165d3-3c7b-45f6-91bf-3f1cc202d896 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:31.547238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:31.607586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4d15f-55c1-47cc-936c-8367be32627f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:31.608181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4d15f-55c1-47cc-936c-8367be32627f service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:31.608455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4d15f-55c1-47cc-936c-8367be32627f service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:31.608723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4d15f-55c1-47cc-936c-8367be32627f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:31.610333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0ab4d15f-55c1-47cc-936c-8367be32627f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:31.720763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4d15f-55c1-47cc-936c-8367be32627f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:31.722642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4d15f-55c1-47cc-936c-8367be32627f service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:31.740708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 721/1446] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:34.289742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a9242-b151-4770-b1bf-2e3f863682ee None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:34.290323 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a9242-b151-4770-b1bf-2e3f863682ee None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:34.290323 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a9242-b151-4770-b1bf-2e3f863682ee None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:34.291647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:53:34.297090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a0d41-c4f7-4765-a06a-406383c798b6 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:34.297506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a0d41-c4f7-4765-a06a-406383c798b6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:34.297622 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a0d41-c4f7-4765-a06a-406383c798b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:34.346678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:34.364746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-642a0d41-c4f7-4765-a06a-406383c798b6 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:34.365997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-642a0d41-c4f7-4765-a06a-406383c798b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:34.462439 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:34.750626 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a9242-b151-4770-b1bf-2e3f863682ee None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:34.752280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ea5a9242-b151-4770-b1bf-2e3f863682ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:34.856685 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a9242-b151-4770-b1bf-2e3f863682ee None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:34.898390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:34.933448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d4807-c7ed-40c9-82e4-d26974f7b264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:34.934071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d4807-c7ed-40c9-82e4-d26974f7b264 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:34.934350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d4807-c7ed-40c9-82e4-d26974f7b264 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:34.934620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d4807-c7ed-40c9-82e4-d26974f7b264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:34.936288 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-dd6d4807-c7ed-40c9-82e4-d26974f7b264 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:35.045053 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d4807-c7ed-40c9-82e4-d26974f7b264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:35.046486 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d4807-c7ed-40c9-82e4-d26974f7b264 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:35.065487 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 727/1449] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:37.338285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d5253-a34a-4bda-bf3a-9c928487a42a None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:37.338860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d5253-a34a-4bda-bf3a-9c928487a42a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:37.338860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d5253-a34a-4bda-bf3a-9c928487a42a None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:37.340228 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:37.345808 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a1ff1-a9f0-4d4d-a5c1-fea588fd32a4 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:37.346222 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a1ff1-a9f0-4d4d-a5c1-fea588fd32a4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:37.346332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a1ff1-a9f0-4d4d-a5c1-fea588fd32a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:37.413720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ec4a1ff1-a9f0-4d4d-a5c1-fea588fd32a4 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:37.414950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ec4a1ff1-a9f0-4d4d-a5c1-fea588fd32a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:37.435259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:37.512061 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:37.797892 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d5253-a34a-4bda-bf3a-9c928487a42a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:37.799451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f48d5253-a34a-4bda-bf3a-9c928487a42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:37.903912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d5253-a34a-4bda-bf3a-9c928487a42a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:37.924600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:37.979364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc7d97-7350-4d1f-959d-49c661d82df0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:37.979963 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc7d97-7350-4d1f-959d-49c661d82df0 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:37.980236 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc7d97-7350-4d1f-959d-49c661d82df0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:37.980502 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc7d97-7350-4d1f-959d-49c661d82df0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:37.982471 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-69dc7d97-7350-4d1f-959d-49c661d82df0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:38.092678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dc7d97-7350-4d1f-959d-49c661d82df0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:38.094020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dc7d97-7350-4d1f-959d-49c661d82df0 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:38.113700 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 724/1452] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:40.341897 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8885b-ca2c-4d3c-8eac-a667845f417f None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.342487 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8885b-ca2c-4d3c-8eac-a667845f417f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.342487 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8885b-ca2c-4d3c-8eac-a667845f417f None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.343807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:40.350449 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995541d-2e5f-4bd6-bada-d00e06c869fa None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.351121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995541d-2e5f-4bd6-bada-d00e06c869fa None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.351265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995541d-2e5f-4bd6-bada-d00e06c869fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.417375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:40.420153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0995541d-2e5f-4bd6-bada-d00e06c869fa None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:40.421520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0995541d-2e5f-4bd6-bada-d00e06c869fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:40.522390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:40.806990 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8885b-ca2c-4d3c-8eac-a667845f417f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:40.808549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7ea8885b-ca2c-4d3c-8eac-a667845f417f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:40.917961 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8885b-ca2c-4d3c-8eac-a667845f417f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:40.967420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:40.991541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9daa98f-33e6-4623-b372-de27f8d90265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:40.992164 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9daa98f-33e6-4623-b372-de27f8d90265 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.992445 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9daa98f-33e6-4623-b372-de27f8d90265 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.992715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9daa98f-33e6-4623-b372-de27f8d90265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.994382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a9daa98f-33e6-4623-b372-de27f8d90265 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:41.107752 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9daa98f-33e6-4623-b372-de27f8d90265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:41.109203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9daa98f-33e6-4623-b372-de27f8d90265 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:41.128544 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 730/1455] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:43.386696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5063fe0-f5be-4d62-92f4-54d6614a31bd None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:43.387286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5063fe0-f5be-4d62-92f4-54d6614a31bd None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:43.387286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5063fe0-f5be-4d62-92f4-54d6614a31bd None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:43.388633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:43.393892 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797b95f-7daf-4816-9a2e-8205e1791b51 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:43.394291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797b95f-7daf-4816-9a2e-8205e1791b51 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:43.394442 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797b95f-7daf-4816-9a2e-8205e1791b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:43.460390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-0797b95f-7daf-4816-9a2e-8205e1791b51 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:43.461607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0797b95f-7daf-4816-9a2e-8205e1791b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:43.491088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:43.559215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:43.847723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5063fe0-f5be-4d62-92f4-54d6614a31bd None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:43.849268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a5063fe0-f5be-4d62-92f4-54d6614a31bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:43.951608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5063fe0-f5be-4d62-92f4-54d6614a31bd None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:43.979695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:44.029065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677560c-9768-40d0-a4b7-0e0edb17eda6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:44.029666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677560c-9768-40d0-a4b7-0e0edb17eda6 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:44.029932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677560c-9768-40d0-a4b7-0e0edb17eda6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:44.030461 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677560c-9768-40d0-a4b7-0e0edb17eda6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:44.032071 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5677560c-9768-40d0-a4b7-0e0edb17eda6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:44.139918 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5677560c-9768-40d0-a4b7-0e0edb17eda6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:44.141355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5677560c-9768-40d0-a4b7-0e0edb17eda6 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:44.159841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 727/1458] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:46.393755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6ee72-2c92-4d3a-8ff3-d0f8cce1111e None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:46.394328 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6ee72-2c92-4d3a-8ff3-d0f8cce1111e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:46.394328 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6ee72-2c92-4d3a-8ff3-d0f8cce1111e None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.395663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:46.400971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675cd19-deab-479a-ad3a-acdcca9deaf0 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:46.401391 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675cd19-deab-479a-ad3a-acdcca9deaf0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:46.401524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675cd19-deab-479a-ad3a-acdcca9deaf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.468537 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a675cd19-deab-479a-ad3a-acdcca9deaf0 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:46.469750 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a675cd19-deab-479a-ad3a-acdcca9deaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:46.478085 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:46.573384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:46.876964 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6ee72-2c92-4d3a-8ff3-d0f8cce1111e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:46.879498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-32e6ee72-2c92-4d3a-8ff3-d0f8cce1111e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:47.025157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6ee72-2c92-4d3a-8ff3-d0f8cce1111e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:47.059325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:47.085923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a726e5f-b47a-4cbe-81dc-a3ffec18478c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:47.095060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a726e5f-b47a-4cbe-81dc-a3ffec18478c service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:47.095060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a726e5f-b47a-4cbe-81dc-a3ffec18478c service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:47.095060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a726e5f-b47a-4cbe-81dc-a3ffec18478c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:47.095060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2a726e5f-b47a-4cbe-81dc-a3ffec18478c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:47.214530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a726e5f-b47a-4cbe-81dc-a3ffec18478c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:47.216156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a726e5f-b47a-4cbe-81dc-a3ffec18478c service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:47.234315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 733/1461] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:49.504032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9e60c-4e52-45f7-bfbc-2d8f8fd19047 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:49.504590 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9e60c-4e52-45f7-bfbc-2d8f8fd19047 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:49.504697 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9e60c-4e52-45f7-bfbc-2d8f8fd19047 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:49.506052 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:49.511546 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297742c8-6284-4e0f-9581-b8d406f5d48b None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:49.511945 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297742c8-6284-4e0f-9581-b8d406f5d48b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:49.512095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297742c8-6284-4e0f-9581-b8d406f5d48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:49.576777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-297742c8-6284-4e0f-9581-b8d406f5d48b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:49.577971 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-297742c8-6284-4e0f-9581-b8d406f5d48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:49.615937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:49.674042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:49.964334 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9e60c-4e52-45f7-bfbc-2d8f8fd19047 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:49.965928 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-92a9e60c-4e52-45f7-bfbc-2d8f8fd19047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:50.069641 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9e60c-4e52-45f7-bfbc-2d8f8fd19047 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:50.107777 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:50.142831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1438f-0495-42cf-852d-afc8ef43f852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:50.144040 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1438f-0495-42cf-852d-afc8ef43f852 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:50.144320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1438f-0495-42cf-852d-afc8ef43f852 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:50.144591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1438f-0495-42cf-852d-afc8ef43f852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:50.146199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3bd1438f-0495-42cf-852d-afc8ef43f852 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:50.255060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1438f-0495-42cf-852d-afc8ef43f852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:50.256415 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1438f-0495-42cf-852d-afc8ef43f852 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:50.276091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 730/1464] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:52.440877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578374f-c21d-4833-8edd-8c42b1ce76c2 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:52.441443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578374f-c21d-4833-8edd-8c42b1ce76c2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:52.441570 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578374f-c21d-4833-8edd-8c42b1ce76c2 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:52.443121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:53:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:52.448340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934320f-c33b-43c8-b5a0-f6b5de00ec8b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:52.448813 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934320f-c33b-43c8-b5a0-f6b5de00ec8b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:52.448925 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934320f-c33b-43c8-b5a0-f6b5de00ec8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:52.516724 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1934320f-c33b-43c8-b5a0-f6b5de00ec8b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:52.518569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1934320f-c33b-43c8-b5a0-f6b5de00ec8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:52.536831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:52.618633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:52.902418 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578374f-c21d-4833-8edd-8c42b1ce76c2 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:52.903999 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3578374f-c21d-4833-8edd-8c42b1ce76c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:53.009196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578374f-c21d-4833-8edd-8c42b1ce76c2 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:53.077758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:53.084283 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4395cd-b47a-4314-95e3-8bfba062cf1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:53.084891 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4395cd-b47a-4314-95e3-8bfba062cf1d service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:53.085171 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4395cd-b47a-4314-95e3-8bfba062cf1d service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:53.085438 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4395cd-b47a-4314-95e3-8bfba062cf1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:53.087060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1c4395cd-b47a-4314-95e3-8bfba062cf1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:53.200562 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4395cd-b47a-4314-95e3-8bfba062cf1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:53.202254 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4395cd-b47a-4314-95e3-8bfba062cf1d service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:53.221628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 736/1467] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:55.419958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3f912-bb36-4217-b24c-ed36d893ed3e None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:55.420528 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3f912-bb36-4217-b24c-ed36d893ed3e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:55.420528 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3f912-bb36-4217-b24c-ed36d893ed3e None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:55.421834 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:55.427892 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e5165-f774-460f-8e2f-964d0cd6d648 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:55.428334 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e5165-f774-460f-8e2f-964d0cd6d648 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:55.428445 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e5165-f774-460f-8e2f-964d0cd6d648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:55.507444 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-0c1e5165-f774-460f-8e2f-964d0cd6d648 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:55.510002 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0c1e5165-f774-460f-8e2f-964d0cd6d648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:55.558231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:55.607026 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:55.892625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3f912-bb36-4217-b24c-ed36d893ed3e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:55.894240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-07a3f912-bb36-4217-b24c-ed36d893ed3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:55.994564 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3f912-bb36-4217-b24c-ed36d893ed3e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:56.043023 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:56.067512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b04c7a-4f29-415d-9d96-4bcecd753d0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:56.068180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b04c7a-4f29-415d-9d96-4bcecd753d0e service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:56.068458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b04c7a-4f29-415d-9d96-4bcecd753d0e service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:56.068725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b04c7a-4f29-415d-9d96-4bcecd753d0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.070318 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-20b04c7a-4f29-415d-9d96-4bcecd753d0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:56.179340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b04c7a-4f29-415d-9d96-4bcecd753d0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:56.180680 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b04c7a-4f29-415d-9d96-4bcecd753d0e service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:56.199027 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 733/1470] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:05.605453 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=22 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:06.196767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=66 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:16.457341 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbbbc5-7017-4351-b7ea-9906099ecef8 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.457922 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbbbc5-7017-4351-b7ea-9906099ecef8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.457922 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbbbc5-7017-4351-b7ea-9906099ecef8 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.459281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 738/1471] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:16.464627 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f13bf-8f5a-41c7-ac41-9909ce11788b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.465084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f13bf-8f5a-41c7-ac41-9909ce11788b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.465192 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f13bf-8f5a-41c7-ac41-9909ce11788b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.532351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-712f13bf-8f5a-41c7-ac41-9909ce11788b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:16.534059 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-712f13bf-8f5a-41c7-ac41-9909ce11788b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:16.631711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 734/1472] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:16.955029 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46655e-3bc9-4e1e-9499-1a9fd68ef8b0 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.955314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46655e-3bc9-4e1e-9499-1a9fd68ef8b0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.955590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46655e-3bc9-4e1e-9499-1a9fd68ef8b0 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.956984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 739/1473] 10.222.0.91 () {54 vars in 961 bytes} [Thu Sep 8 13:54:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:16.962496 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a86f17-6009-443c-a04f-99812c02ec39 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.962776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a86f17-6009-443c-a04f-99812c02ec39 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.963067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a86f17-6009-443c-a04f-99812c02ec39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.040994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d8a86f17-6009-443c-a04f-99812c02ec39 None None] MFA Rules not processed for user `01b060c70e1b440186eeac37d9c34567`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:17.042204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d8a86f17-6009-443c-a04f-99812c02ec39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:17.062603 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:17.117524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 735/1474] 10.222.0.91 () {58 vars in 1058 bytes} [Thu Sep 8 13:54:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:17.125303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46655e-3bc9-4e1e-9499-1a9fd68ef8b0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:17.127395 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5c46655e-3bc9-4e1e-9499-1a9fd68ef8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:17.234697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46655e-3bc9-4e1e-9499-1a9fd68ef8b0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:17.309331 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ba4ae-f43c-4886-8f91-79d88af2994c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b060c70e1b440186eeac37d9c34567', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:17.309931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ba4ae-f43c-4886-8f91-79d88af2994c service glance] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.310206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ba4ae-f43c-4886-8f91-79d88af2994c service glance] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.310495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ba4ae-f43c-4886-8f91-79d88af2994c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.312083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-aa0ba4ae-f43c-4886-8f91-79d88af2994c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:17.366652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:17.421082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ba4ae-f43c-4886-8f91-79d88af2994c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:17.422509 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ba4ae-f43c-4886-8f91-79d88af2994c service glance] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:17.441516 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 740/1475] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:19.681367 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde2a6d-9ba6-4812-b68f-bdde2f7be208 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:19.681971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde2a6d-9ba6-4812-b68f-bdde2f7be208 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:19.681971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde2a6d-9ba6-4812-b68f-bdde2f7be208 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:19.683297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 736/1476] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:19.689249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206d2f4-f523-46e5-aa46-c19032d915cb None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:19.689710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206d2f4-f523-46e5-aa46-c19032d915cb None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:19.689812 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206d2f4-f523-46e5-aa46-c19032d915cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:19.757397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3206d2f4-f523-46e5-aa46-c19032d915cb None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:19.758557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3206d2f4-f523-46e5-aa46-c19032d915cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:19.854365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 741/1477] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:20.130038 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde2a6d-9ba6-4812-b68f-bdde2f7be208 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:20.131609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7bde2a6d-9ba6-4812-b68f-bdde2f7be208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:20.258877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde2a6d-9ba6-4812-b68f-bdde2f7be208 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:20.310207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:20.355104 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f508c-d171-4e58-b2bf-12f76cd3bf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:20.355702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f508c-d171-4e58-b2bf-12f76cd3bf02 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:20.355981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f508c-d171-4e58-b2bf-12f76cd3bf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:20.356253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f508c-d171-4e58-b2bf-12f76cd3bf02 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:20.356888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f508c-d171-4e58-b2bf-12f76cd3bf02 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:20.358703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: WARNING py.warnings [None req-aa4f508c-d171-4e58-b2bf-12f76cd3bf02 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:54:20.358703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: warnings.warn(msg) Sep 08 13:54:20.358703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: Sep 08 13:54:20.359874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f508c-d171-4e58-b2bf-12f76cd3bf02 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:20.378631 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 737/1478] 10.222.0.91 () {56 vars in 1108 bytes} [Thu Sep 8 13:54:20 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:22.595441 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2014db6-23c2-4f45-9183-684991c70d6f None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:22.596033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2014db6-23c2-4f45-9183-684991c70d6f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:22.596033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2014db6-23c2-4f45-9183-684991c70d6f None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:22.597465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 742/1479] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:22.602865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1d1c3-a478-4ba8-995e-e339c1e7e25e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:22.603316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1d1c3-a478-4ba8-995e-e339c1e7e25e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:22.603442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1d1c3-a478-4ba8-995e-e339c1e7e25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:22.669902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e2b1d1c3-a478-4ba8-995e-e339c1e7e25e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:22.671161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e2b1d1c3-a478-4ba8-995e-e339c1e7e25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:22.767359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 738/1480] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:23.069922 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2014db6-23c2-4f45-9183-684991c70d6f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:23.071476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e2014db6-23c2-4f45-9183-684991c70d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:23.081781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:23.225841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2014db6-23c2-4f45-9183-684991c70d6f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:23.316309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:23.348615 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd68ee-4b7c-447f-a07c-2327f746d4fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:23.349236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd68ee-4b7c-447f-a07c-2327f746d4fd service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:23.349516 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd68ee-4b7c-447f-a07c-2327f746d4fd service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:23.349788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd68ee-4b7c-447f-a07c-2327f746d4fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:23.351467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a3bd68ee-4b7c-447f-a07c-2327f746d4fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:23.445549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd68ee-4b7c-447f-a07c-2327f746d4fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:23.447184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd68ee-4b7c-447f-a07c-2327f746d4fd service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:23.466052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 743/1481] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:25.842392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f86847-2408-4c0d-80d3-ccfa7c4126a2 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:25.843303 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f86847-2408-4c0d-80d3-ccfa7c4126a2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:25.843603 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f86847-2408-4c0d-80d3-ccfa7c4126a2 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:25.845501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 739/1482] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:54:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:25.851645 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb208356-9bc1-4f37-85aa-9ab23d3c6d95 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:25.852115 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb208356-9bc1-4f37-85aa-9ab23d3c6d95 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:25.852217 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb208356-9bc1-4f37-85aa-9ab23d3c6d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:25.922411 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-cb208356-9bc1-4f37-85aa-9ab23d3c6d95 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:25.924667 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-cb208356-9bc1-4f37-85aa-9ab23d3c6d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:25.981157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:26.040288 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 744/1483] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:26.337307 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f86847-2408-4c0d-80d3-ccfa7c4126a2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:26.339484 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-14f86847-2408-4c0d-80d3-ccfa7c4126a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:26.363407 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:26.448305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f86847-2408-4c0d-80d3-ccfa7c4126a2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:26.529597 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279a77d-eb7c-41d1-b7d1-4663f965ea0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:26.530199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279a77d-eb7c-41d1-b7d1-4663f965ea0f service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:26.530480 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279a77d-eb7c-41d1-b7d1-4663f965ea0f service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:26.530754 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279a77d-eb7c-41d1-b7d1-4663f965ea0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:26.532422 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f279a77d-eb7c-41d1-b7d1-4663f965ea0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:26.614920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:26.650416 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279a77d-eb7c-41d1-b7d1-4663f965ea0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:26.651882 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279a77d-eb7c-41d1-b7d1-4663f965ea0f service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:26.788711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 740/1484] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:29.000580 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146e4eb-3aa1-48bf-902f-441ef8268077 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:29.001661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146e4eb-3aa1-48bf-902f-441ef8268077 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:29.001754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146e4eb-3aa1-48bf-902f-441ef8268077 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:29.003099 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 745/1485] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:54:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:29.008693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d006b63-5339-4377-9100-3e5c0210dcf7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:29.009135 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d006b63-5339-4377-9100-3e5c0210dcf7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:29.009231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d006b63-5339-4377-9100-3e5c0210dcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:29.076335 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4d006b63-5339-4377-9100-3e5c0210dcf7 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:29.077526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4d006b63-5339-4377-9100-3e5c0210dcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:29.182258 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 741/1486] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:54:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:29.475731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146e4eb-3aa1-48bf-902f-441ef8268077 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:29.477271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d146e4eb-3aa1-48bf-902f-441ef8268077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:29.581465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146e4eb-3aa1-48bf-902f-441ef8268077 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:29.652261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:29.653753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5108579e-38d4-49f8-ba54-6f2cf04841ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:29.654673 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5108579e-38d4-49f8-ba54-6f2cf04841ec service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:29.654984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5108579e-38d4-49f8-ba54-6f2cf04841ec service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:29.655259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5108579e-38d4-49f8-ba54-6f2cf04841ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:29.656859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5108579e-38d4-49f8-ba54-6f2cf04841ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:29.772294 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5108579e-38d4-49f8-ba54-6f2cf04841ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:29.773638 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5108579e-38d4-49f8-ba54-6f2cf04841ec service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:29.792350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 746/1487] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:32.176722 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4281a72-75e5-4c0b-9dec-794a2cc181ca None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:32.177348 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4281a72-75e5-4c0b-9dec-794a2cc181ca None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:32.177348 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4281a72-75e5-4c0b-9dec-794a2cc181ca None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:32.178590 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 742/1488] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:32.184088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1205a4f-2e43-470c-a8b9-0c0194036da9 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:32.184536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1205a4f-2e43-470c-a8b9-0c0194036da9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:32.184686 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1205a4f-2e43-470c-a8b9-0c0194036da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:32.249411 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b1205a4f-2e43-470c-a8b9-0c0194036da9 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:32.251281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b1205a4f-2e43-470c-a8b9-0c0194036da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:32.305409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:32.351010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 747/1489] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:32.653418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4281a72-75e5-4c0b-9dec-794a2cc181ca None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:32.655314 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d4281a72-75e5-4c0b-9dec-794a2cc181ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:32.693697 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:32.762963 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4281a72-75e5-4c0b-9dec-794a2cc181ca None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:32.842399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefb8ce-9cb3-449c-8df9-eb22fac42e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:32.843023 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefb8ce-9cb3-449c-8df9-eb22fac42e8d service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:32.843303 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefb8ce-9cb3-449c-8df9-eb22fac42e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:32.843574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefb8ce-9cb3-449c-8df9-eb22fac42e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:32.845659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-eeefb8ce-9cb3-449c-8df9-eb22fac42e8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:32.926574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:32.954800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefb8ce-9cb3-449c-8df9-eb22fac42e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:32.956283 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefb8ce-9cb3-449c-8df9-eb22fac42e8d service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:32.974871 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 743/1490] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:35.629109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381968ec-450d-45fa-8884-0ebd713d6fe1 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:35.629649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381968ec-450d-45fa-8884-0ebd713d6fe1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:35.629649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381968ec-450d-45fa-8884-0ebd713d6fe1 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:35.631047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 748/1491] 10.222.0.91 () {54 vars in 927 bytes} [Thu Sep 8 13:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:35.636398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699c0fa-4e3f-49e7-ac13-29f0158da855 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:35.636790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699c0fa-4e3f-49e7-ac13-29f0158da855 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:35.636940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699c0fa-4e3f-49e7-ac13-29f0158da855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:35.705828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c699c0fa-4e3f-49e7-ac13-29f0158da855 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:35.707077 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c699c0fa-4e3f-49e7-ac13-29f0158da855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:35.803003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 744/1492] 10.222.0.91 () {58 vars in 1024 bytes} [Thu Sep 8 13:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:36.092441 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381968ec-450d-45fa-8884-0ebd713d6fe1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:36.093995 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-381968ec-450d-45fa-8884-0ebd713d6fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:36.196933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381968ec-450d-45fa-8884-0ebd713d6fe1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:36.271091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e2a24-da83-4ab9-a98c-8f3591cdb9ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:36.271711 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e2a24-da83-4ab9-a98c-8f3591cdb9ec service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:36.271985 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e2a24-da83-4ab9-a98c-8f3591cdb9ec service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:36.272256 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e2a24-da83-4ab9-a98c-8f3591cdb9ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:36.273878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5a3e2a24-da83-4ab9-a98c-8f3591cdb9ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:36.282658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:36.381673 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e2a24-da83-4ab9-a98c-8f3591cdb9ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:36.383460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e2a24-da83-4ab9-a98c-8f3591cdb9ec service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:36.402623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 749/1493] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:45.803982 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=82 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:46.400317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=50 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:03.455145 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746b3b0-5324-4ec4-8f36-ede615c0ac76 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.455785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746b3b0-5324-4ec4-8f36-ede615c0ac76 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.455896 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746b3b0-5324-4ec4-8f36-ede615c0ac76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.539972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e746b3b0-5324-4ec4-8f36-ede615c0ac76 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:03.541211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e746b3b0-5324-4ec4-8f36-ede615c0ac76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:03.650159 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 745/1494] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:03.665624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae9c06-eb32-43bf-92d7-384d0fd41bc0 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.667785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae9c06-eb32-43bf-92d7-384d0fd41bc0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.667785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae9c06-eb32-43bf-92d7-384d0fd41bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.771269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-85ae9c06-eb32-43bf-92d7-384d0fd41bc0 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:03.772774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-85ae9c06-eb32-43bf-92d7-384d0fd41bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:03.877761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 750/1495] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:03.882476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746b3b0-5324-4ec4-8f36-ede615c0ac76 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:03.883708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e746b3b0-5324-4ec4-8f36-ede615c0ac76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:03.895303 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746b3b0-5324-4ec4-8f36-ede615c0ac76 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:03.962398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:04.004418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dc1bc-746a-4d55-b5c4-69327aeffceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:04.005016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dc1bc-746a-4d55-b5c4-69327aeffceb admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:04.005351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dc1bc-746a-4d55-b5c4-69327aeffceb admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:04.005623 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dc1bc-746a-4d55-b5c4-69327aeffceb admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:04.006285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5dc1bc-746a-4d55-b5c4-69327aeffceb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:56:04.006681 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5dc1bc-746a-4d55-b5c4-69327aeffceb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:04.007699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: WARNING py.warnings [None req-6a5dc1bc-746a-4d55-b5c4-69327aeffceb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:56:04.007699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: warnings.warn(msg) Sep 08 13:56:04.007699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: Sep 08 13:56:04.008582 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5dc1bc-746a-4d55-b5c4-69327aeffceb admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:04.020616 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 746/1496] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 13:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:56:04.029527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae9c06-eb32-43bf-92d7-384d0fd41bc0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:04.043931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae9c06-eb32-43bf-92d7-384d0fd41bc0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:04.114505 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:04.153502 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cc3da-d73c-45e4-a9e6-8f93a7d5448f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:04.154137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cc3da-d73c-45e4-a9e6-8f93a7d5448f admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:04.154412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cc3da-d73c-45e4-a9e6-8f93a7d5448f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:04.154682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cc3da-d73c-45e4-a9e6-8f93a7d5448f admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:04.155346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6cc3da-d73c-45e4-a9e6-8f93a7d5448f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:04.157174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: WARNING py.warnings [None req-1a6cc3da-d73c-45e4-a9e6-8f93a7d5448f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:56:04.157174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: warnings.warn(msg) Sep 08 13:56:04.157174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: Sep 08 13:56:04.158369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6cc3da-d73c-45e4-a9e6-8f93a7d5448f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:04.295145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 751/1497] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:56:04 2022] POST /identity/v3/projects => generated 375 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:04.299943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dc1bc-746a-4d55-b5c4-69327aeffceb admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:04.303890 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6a5dc1bc-746a-4d55-b5c4-69327aeffceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:04.342874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dc1bc-746a-4d55-b5c4-69327aeffceb admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:04.450372 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b1d01-0c5e-4904-a9ac-6043564d6fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:04.450996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b1d01-0c5e-4904-a9ac-6043564d6fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:04.451273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b1d01-0c5e-4904-a9ac-6043564d6fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:04.451539 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b1d01-0c5e-4904-a9ac-6043564d6fc8 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:04.452153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b1d01-0c5e-4904-a9ac-6043564d6fc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:04.453180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: WARNING py.warnings [None req-287b1d01-0c5e-4904-a9ac-6043564d6fc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:56:04.453180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: warnings.warn(msg) Sep 08 13:56:04.453180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: Sep 08 13:56:04.454393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b1d01-0c5e-4904-a9ac-6043564d6fc8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:04.569566 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 747/1498] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:56:04 2022] POST /identity/v3/users => generated 288 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:56:04.573805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cc3da-d73c-45e4-a9e6-8f93a7d5448f admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:04.575437 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1a6cc3da-d73c-45e4-a9e6-8f93a7d5448f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:04.641967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cc3da-d73c-45e4-a9e6-8f93a7d5448f admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:04.747569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:04.768279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d71e0-cfc8-4f1e-a0ab-124707aace1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:04.772396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d71e0-cfc8-4f1e-a0ab-124707aace1c admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:04.772396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d71e0-cfc8-4f1e-a0ab-124707aace1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:04.772396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d71e0-cfc8-4f1e-a0ab-124707aace1c admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:04.772396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06d71e0-cfc8-4f1e-a0ab-124707aace1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:04.772396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: WARNING py.warnings [None req-d06d71e0-cfc8-4f1e-a0ab-124707aace1c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:56:04.772396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: warnings.warn(msg) Sep 08 13:56:04.772396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: Sep 08 13:56:04.772964 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06d71e0-cfc8-4f1e-a0ab-124707aace1c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:04.786452 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 752/1499] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:04.791295 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b1d01-0c5e-4904-a9ac-6043564d6fc8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:04.801823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b1d01-0c5e-4904-a9ac-6043564d6fc8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:04.815369 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151cf87-e62a-4632-8dd4-cf2055ebf781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:04.815369 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151cf87-e62a-4632-8dd4-cf2055ebf781 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:04.815369 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151cf87-e62a-4632-8dd4-cf2055ebf781 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:04.815369 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151cf87-e62a-4632-8dd4-cf2055ebf781 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:04.815781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1151cf87-e62a-4632-8dd4-cf2055ebf781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:04.815781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: WARNING py.warnings [None req-1151cf87-e62a-4632-8dd4-cf2055ebf781 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:56:04.815781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: warnings.warn(msg) Sep 08 13:56:04.815781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: Sep 08 13:56:04.815781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1151cf87-e62a-4632-8dd4-cf2055ebf781 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:04.826606 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 748/1500] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:56:04.831058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d71e0-cfc8-4f1e-a0ab-124707aace1c admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:04.842602 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d71e0-cfc8-4f1e-a0ab-124707aace1c admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:04.925703 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb226325-4bc2-416d-9f98-9f5d2112274e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:04.926367 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb226325-4bc2-416d-9f98-9f5d2112274e admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:04.926643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb226325-4bc2-416d-9f98-9f5d2112274e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:04.926936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb226325-4bc2-416d-9f98-9f5d2112274e admin admin] PATH_INFO: `/v3/projects/f92c25d0485946a09017662e2419d6ed/users/9b01b0c891584ba7bbf3e6f6573cf503/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:04.954351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb226325-4bc2-416d-9f98-9f5d2112274e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f92c25d0485946a09017662e2419d6ed, user_id=9b01b0c891584ba7bbf3e6f6573cf503, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:04.955425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: WARNING py.warnings [None req-bb226325-4bc2-416d-9f98-9f5d2112274e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:56:04.955425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: warnings.warn(msg) Sep 08 13:56:04.955425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: Sep 08 13:56:04.957364 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb226325-4bc2-416d-9f98-9f5d2112274e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:05.029585 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 753/1501] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:56:04 2022] PUT /identity/v3/projects/f92c25d0485946a09017662e2419d6ed/users/9b01b0c891584ba7bbf3e6f6573cf503/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:05.041926 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac1ef9-d82e-4d93-bbdc-e820662e8403 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:05.042212 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac1ef9-d82e-4d93-bbdc-e820662e8403 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:05.042487 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac1ef9-d82e-4d93-bbdc-e820662e8403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:05.072832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7cac1ef9-d82e-4d93-bbdc-e820662e8403 None None] MFA Rules not processed for user `9b01b0c891584ba7bbf3e6f6573cf503`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:05.074046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7cac1ef9-d82e-4d93-bbdc-e820662e8403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:05.079892 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:05.150402 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 749/1502] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:56:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:56:05.164109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66dc65-634b-4b78-a8fb-187910bdd34e None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:05.164525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66dc65-634b-4b78-a8fb-187910bdd34e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:05.164966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66dc65-634b-4b78-a8fb-187910bdd34e None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:05.167248 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 754/1503] 10.222.0.91 () {54 vars in 964 bytes} [Thu Sep 8 13:56:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:56:05.174492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0e7a8-fd88-4fe7-bc87-a8d33f7b9d70 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:05.174782 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0e7a8-fd88-4fe7-bc87-a8d33f7b9d70 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:05.175804 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0e7a8-fd88-4fe7-bc87-a8d33f7b9d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:05.266352 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-5da0e7a8-fd88-4fe7-bc87-a8d33f7b9d70 None None] MFA Rules not processed for user `ca225b99584e40f4a4065bf10264974d`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:05.267754 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5da0e7a8-fd88-4fe7-bc87-a8d33f7b9d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:05.349475 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 750/1504] 10.222.0.91 () {58 vars in 1061 bytes} [Thu Sep 8 13:56:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:05.360653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66dc65-634b-4b78-a8fb-187910bdd34e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:05.372771 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66dc65-634b-4b78-a8fb-187910bdd34e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:05.378720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10ab5f-0161-4997-9e66-5a086cea80fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:05.379351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10ab5f-0161-4997-9e66-5a086cea80fa service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:05.379627 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10ab5f-0161-4997-9e66-5a086cea80fa service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:05.379894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10ab5f-0161-4997-9e66-5a086cea80fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:05.388966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10ab5f-0161-4997-9e66-5a086cea80fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:05.390404 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10ab5f-0161-4997-9e66-5a086cea80fa service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:05.400462 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 755/1505] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:05.450869 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11082b-8b46-4a70-965f-fc947ca71ce2 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:05.451185 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11082b-8b46-4a70-965f-fc947ca71ce2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:05.451472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11082b-8b46-4a70-965f-fc947ca71ce2 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:05.452767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 751/1506] 10.222.0.91 () {52 vars in 830 bytes} [Thu Sep 8 13:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:56:05.456940 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10ab5f-0161-4997-9e66-5a086cea80fa service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:05.522152 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10ab5f-0161-4997-9e66-5a086cea80fa service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:05.526851 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16843b9-1a62-4c78-88ba-9d4c4b90860f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:05.527766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16843b9-1a62-4c78-88ba-9d4c4b90860f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:05.528047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16843b9-1a62-4c78-88ba-9d4c4b90860f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:05.528325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16843b9-1a62-4c78-88ba-9d4c4b90860f admin admin] PATH_INFO: `/v3/users/9b01b0c891584ba7bbf3e6f6573cf503` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:05.529476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16843b9-1a62-4c78-88ba-9d4c4b90860f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b01b0c891584ba7bbf3e6f6573cf503)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:05.530549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: WARNING py.warnings [None req-a16843b9-1a62-4c78-88ba-9d4c4b90860f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:56:05.530549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: warnings.warn(msg) Sep 08 13:56:05.530549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: Sep 08 13:56:05.532320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16843b9-1a62-4c78-88ba-9d4c4b90860f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:05.761547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:05.773208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a16843b9-1a62-4c78-88ba-9d4c4b90860f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b01b0c891584ba7bbf3e6f6573cf503'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:05.885678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a16843b9-1a62-4c78-88ba-9d4c4b90860f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b01b0c891584ba7bbf3e6f6573cf503'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:05.885860 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a16843b9-1a62-4c78-88ba-9d4c4b90860f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b01b0c891584ba7bbf3e6f6573cf503'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:05.902724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a16843b9-1a62-4c78-88ba-9d4c4b90860f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b01b0c891584ba7bbf3e6f6573cf503'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:05.932439 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a16843b9-1a62-4c78-88ba-9d4c4b90860f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b01b0c891584ba7bbf3e6f6573cf503'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:05.944670 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 756/1507] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 13:56:05 2022] DELETE /identity/v3/users/9b01b0c891584ba7bbf3e6f6573cf503 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:56:05.954193 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e307bd-255e-4b10-a203-df63808538ba None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:05.954971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e307bd-255e-4b10-a203-df63808538ba None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:05.954971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e307bd-255e-4b10-a203-df63808538ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:05.984907 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-65e307bd-255e-4b10-a203-df63808538ba None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:05.986111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-65e307bd-255e-4b10-a203-df63808538ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:06.027549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:06.047388 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 752/1508] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:06.057421 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16843b9-1a62-4c78-88ba-9d4c4b90860f admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:06.059065 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a16843b9-1a62-4c78-88ba-9d4c4b90860f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:06.148231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16843b9-1a62-4c78-88ba-9d4c4b90860f admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:06.151981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffe87f-6b06-41f8-bf09-8d009778051a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:06.152903 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffe87f-6b06-41f8-bf09-8d009778051a service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:06.153188 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffe87f-6b06-41f8-bf09-8d009778051a service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:06.153455 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffe87f-6b06-41f8-bf09-8d009778051a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:06.164325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffe87f-6b06-41f8-bf09-8d009778051a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:06.165773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffe87f-6b06-41f8-bf09-8d009778051a service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:06.176144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 757/1509] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:06.711815 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e307bd-255e-4b10-a203-df63808538ba None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:06.713338 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-65e307bd-255e-4b10-a203-df63808538ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:06.726094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e307bd-255e-4b10-a203-df63808538ba None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:06.738050 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a165ee-31fe-4ea4-a7a8-f3dc11f648e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:06.738914 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a165ee-31fe-4ea4-a7a8-f3dc11f648e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:06.739329 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a165ee-31fe-4ea4-a7a8-f3dc11f648e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:06.739908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a165ee-31fe-4ea4-a7a8-f3dc11f648e1 admin admin] PATH_INFO: `/v3/projects/f92c25d0485946a09017662e2419d6ed` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:06.741360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a165ee-31fe-4ea4-a7a8-f3dc11f648e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f92c25d0485946a09017662e2419d6ed)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:06.743953 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: WARNING py.warnings [None req-43a165ee-31fe-4ea4-a7a8-f3dc11f648e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:56:06.743953 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: warnings.warn(msg) Sep 08 13:56:06.743953 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: Sep 08 13:56:06.747265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a165ee-31fe-4ea4-a7a8-f3dc11f648e1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:06.852265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-43a165ee-31fe-4ea4-a7a8-f3dc11f648e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f92c25d0485946a09017662e2419d6ed', 'request_id': 'req-43a165ee-31fe-4ea4-a7a8-f3dc11f648e1'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:06.858379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-43a165ee-31fe-4ea4-a7a8-f3dc11f648e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f92c25d0485946a09017662e2419d6ed', 'request_id': 'req-43a165ee-31fe-4ea4-a7a8-f3dc11f648e1'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:06.872326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-43a165ee-31fe-4ea4-a7a8-f3dc11f648e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f92c25d0485946a09017662e2419d6ed', 'request_id': 'req-43a165ee-31fe-4ea4-a7a8-f3dc11f648e1'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:06.959006 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-43a165ee-31fe-4ea4-a7a8-f3dc11f648e1 admin admin] The token cache is being invalidate because project f92c25d0485946a09017662e2419d6ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:56:06.959664 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-43a165ee-31fe-4ea4-a7a8-f3dc11f648e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:06.959962 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-43a165ee-31fe-4ea4-a7a8-f3dc11f648e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:56:06.961728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 753/1510] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 13:56:06 2022] DELETE /identity/v3/projects/f92c25d0485946a09017662e2419d6ed => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:56:16.172199 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:DELETE=5,keystone:SELECT=49,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:16.955456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:40.661542 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11980262-ee04-46cf-9e97-528145385577 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:40.662152 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11980262-ee04-46cf-9e97-528145385577 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:40.662152 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11980262-ee04-46cf-9e97-528145385577 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:40.663467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:57:40.668522 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673b6d0-2b26-40dd-a60c-d2ee5557ce3d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:40.674123 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673b6d0-2b26-40dd-a60c-d2ee5557ce3d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:40.674123 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4673b6d0-2b26-40dd-a60c-d2ee5557ce3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:40.708894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4673b6d0-2b26-40dd-a60c-d2ee5557ce3d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:40.710098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4673b6d0-2b26-40dd-a60c-d2ee5557ce3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:40.768936 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:57:40.922910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11980262-ee04-46cf-9e97-528145385577 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:40.924699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-11980262-ee04-46cf-9e97-528145385577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:41.033429 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11980262-ee04-46cf-9e97-528145385577 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:41.039004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc076ae-bc1e-4e86-af40-5fd5fac3f4ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:41.039616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc076ae-bc1e-4e86-af40-5fd5fac3f4ea service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:41.039920 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc076ae-bc1e-4e86-af40-5fd5fac3f4ea service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:41.040188 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc076ae-bc1e-4e86-af40-5fd5fac3f4ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:41.048760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc076ae-bc1e-4e86-af40-5fd5fac3f4ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:41.050343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc076ae-bc1e-4e86-af40-5fd5fac3f4ea service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:41.060511 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 759/1513] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:42.167408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c56de0-529a-4ad4-859b-bff231202d57 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:42.168021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c56de0-529a-4ad4-859b-bff231202d57 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:42.168021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c56de0-529a-4ad4-859b-bff231202d57 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:42.169339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:57:42.176208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542463d8-b476-40f1-80dd-74cb2f208d72 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:42.176772 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542463d8-b476-40f1-80dd-74cb2f208d72 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:42.176772 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542463d8-b476-40f1-80dd-74cb2f208d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:42.197652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-542463d8-b476-40f1-80dd-74cb2f208d72 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:42.198915 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-542463d8-b476-40f1-80dd-74cb2f208d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:42.208189 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:42.365682 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c56de0-529a-4ad4-859b-bff231202d57 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:42.380726 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c56de0-529a-4ad4-859b-bff231202d57 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:42.386319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be23bd-89ba-4cdc-b035-3321dd2bcbee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:42.386944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be23bd-89ba-4cdc-b035-3321dd2bcbee service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:42.387226 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be23bd-89ba-4cdc-b035-3321dd2bcbee service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:42.387496 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be23bd-89ba-4cdc-b035-3321dd2bcbee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:42.396510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be23bd-89ba-4cdc-b035-3321dd2bcbee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:42.399700 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be23bd-89ba-4cdc-b035-3321dd2bcbee service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:42.410030 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 756/1516] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:42.491693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db386d-a312-4198-987f-ef1f1dd3e8ba None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:42.491982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db386d-a312-4198-987f-ef1f1dd3e8ba None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:42.492261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db386d-a312-4198-987f-ef1f1dd3e8ba None None] PATH_INFO: `/v3` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:42.493646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 761/1517] 10.222.0.105 () {54 vars in 926 bytes} [Thu Sep 8 13:57:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:57:42.500978 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696805b0-cf31-4973-b243-709dbf516b2a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:42.501264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696805b0-cf31-4973-b243-709dbf516b2a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:42.501547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696805b0-cf31-4973-b243-709dbf516b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:42.559179 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-696805b0-cf31-4973-b243-709dbf516b2a None None] MFA Rules not processed for user `f9506139d5744fe69153e7e30a05b436`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:42.560401 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-696805b0-cf31-4973-b243-709dbf516b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:42.633920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 757/1518] 10.222.0.105 () {58 vars in 1016 bytes} [Thu Sep 8 13:57:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:51.682628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f1357-80a1-42f7-86ab-9b8fa8d7f697 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:51.683225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f1357-80a1-42f7-86ab-9b8fa8d7f697 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:51.683341 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f1357-80a1-42f7-86ab-9b8fa8d7f697 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:51.685882 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 762/1519] 10.222.0.91 () {54 vars in 961 bytes} [Thu Sep 8 13:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:57:51.690287 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e421fcb-23bb-4edb-b4bc-e1f85768a1da None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:51.690733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e421fcb-23bb-4edb-b4bc-e1f85768a1da None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:51.690828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e421fcb-23bb-4edb-b4bc-e1f85768a1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:51.711128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1e421fcb-23bb-4edb-b4bc-e1f85768a1da None None] MFA Rules not processed for user `f9506139d5744fe69153e7e30a05b436`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:51.712451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1e421fcb-23bb-4edb-b4bc-e1f85768a1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:51.720932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 758/1520] 10.222.0.91 () {58 vars in 1058 bytes} [Thu Sep 8 13:57:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:51.729220 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f1357-80a1-42f7-86ab-9b8fa8d7f697 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:51.742850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f1357-80a1-42f7-86ab-9b8fa8d7f697 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:51.748075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7681f4-f088-4afb-8888-14a81afbe359 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9506139d5744fe69153e7e30a05b436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:51.748726 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7681f4-f088-4afb-8888-14a81afbe359 service placement] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:51.749005 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7681f4-f088-4afb-8888-14a81afbe359 service placement] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:51.749276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7681f4-f088-4afb-8888-14a81afbe359 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:51.758286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7681f4-f088-4afb-8888-14a81afbe359 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:51.759889 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7681f4-f088-4afb-8888-14a81afbe359 service placement] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:51.769879 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 763/1521] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:53.920283 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa4578-7231-451f-be35-3c82a69c4942 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:53.920940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa4578-7231-451f-be35-3c82a69c4942 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:53.920940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa4578-7231-451f-be35-3c82a69c4942 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:53.922215 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:57:53.928839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c13915-a799-4414-bff7-1dd512401d80 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:53.929316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c13915-a799-4414-bff7-1dd512401d80 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:53.929436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c13915-a799-4414-bff7-1dd512401d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:53.949671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-67c13915-a799-4414-bff7-1dd512401d80 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:53.950861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-67c13915-a799-4414-bff7-1dd512401d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:53.958601 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:54.111260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa4578-7231-451f-be35-3c82a69c4942 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:54.126091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa4578-7231-451f-be35-3c82a69c4942 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:54.131421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ac66d-c96c-4b44-be6c-6e4fae1ebcfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:54.132003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ac66d-c96c-4b44-be6c-6e4fae1ebcfa service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:54.132275 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ac66d-c96c-4b44-be6c-6e4fae1ebcfa service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:54.132543 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ac66d-c96c-4b44-be6c-6e4fae1ebcfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:54.141843 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ac66d-c96c-4b44-be6c-6e4fae1ebcfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:54.143644 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ac66d-c96c-4b44-be6c-6e4fae1ebcfa service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:54.153412 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 760/1524] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:55.229094 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7789c8-6943-4ed4-86d8-e71b5382f93d None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:55.229667 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7789c8-6943-4ed4-86d8-e71b5382f93d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:55.229667 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7789c8-6943-4ed4-86d8-e71b5382f93d None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:55.231327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:57:55.240491 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0b8f1-b853-43d0-91a5-962c067b031b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:55.241017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0b8f1-b853-43d0-91a5-962c067b031b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:55.241017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0b8f1-b853-43d0-91a5-962c067b031b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:55.260291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-bce0b8f1-b853-43d0-91a5-962c067b031b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:55.261528 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bce0b8f1-b853-43d0-91a5-962c067b031b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:55.269328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:57:55.425383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7789c8-6943-4ed4-86d8-e71b5382f93d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:55.439729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7789c8-6943-4ed4-86d8-e71b5382f93d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:55.445072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331b3d2-8632-497d-8af5-d0402cb2a43c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:55.445660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331b3d2-8632-497d-8af5-d0402cb2a43c service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:55.445936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331b3d2-8632-497d-8af5-d0402cb2a43c service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:55.446205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331b3d2-8632-497d-8af5-d0402cb2a43c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:55.454354 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331b3d2-8632-497d-8af5-d0402cb2a43c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:55.455952 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331b3d2-8632-497d-8af5-d0402cb2a43c service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:55.465784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 766/1527] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:59.685944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488ad6d-5f10-4eb7-ae7c-81bdd7435ec3 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:59.686505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488ad6d-5f10-4eb7-ae7c-81bdd7435ec3 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:59.686505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488ad6d-5f10-4eb7-ae7c-81bdd7435ec3 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:59.687910 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:57:59.693787 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac86d7-2b60-43ed-808f-8f94e939ced1 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:59.694197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac86d7-2b60-43ed-808f-8f94e939ced1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:59.694396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac86d7-2b60-43ed-808f-8f94e939ced1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:59.713448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3cac86d7-2b60-43ed-808f-8f94e939ced1 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:59.714783 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3cac86d7-2b60-43ed-808f-8f94e939ced1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:59.722603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:58:00.011365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488ad6d-5f10-4eb7-ae7c-81bdd7435ec3 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:00.013380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f488ad6d-5f10-4eb7-ae7c-81bdd7435ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:00.024672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:00.074286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488ad6d-5f10-4eb7-ae7c-81bdd7435ec3 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:00.077775 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ec1cf-f755-451c-85ed-c61fc12f6681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:00.078370 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ec1cf-f755-451c-85ed-c61fc12f6681 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:00.078648 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ec1cf-f755-451c-85ed-c61fc12f6681 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:00.078946 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ec1cf-f755-451c-85ed-c61fc12f6681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:00.087731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ec1cf-f755-451c-85ed-c61fc12f6681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:00.089147 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ec1cf-f755-451c-85ed-c61fc12f6681 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:00.098876 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 763/1530] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:02.246911 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be03bc-6672-480e-b8f4-e6a8bf383d84 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:02.247448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be03bc-6672-480e-b8f4-e6a8bf383d84 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:02.247565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be03bc-6672-480e-b8f4-e6a8bf383d84 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:02.248865 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:58:02.254252 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d925ad-882c-41ad-93b3-957ee4482279 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:02.254661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d925ad-882c-41ad-93b3-957ee4482279 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:02.254777 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d925ad-882c-41ad-93b3-957ee4482279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:02.274458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-69d925ad-882c-41ad-93b3-957ee4482279 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:58:02.275705 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-69d925ad-882c-41ad-93b3-957ee4482279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:02.288572 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:58:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:58:02.599236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be03bc-6672-480e-b8f4-e6a8bf383d84 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:02.603072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c0be03bc-6672-480e-b8f4-e6a8bf383d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:02.617951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be03bc-6672-480e-b8f4-e6a8bf383d84 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:02.621553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7f296-2245-4818-b363-4c1afae55289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:02.622225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7f296-2245-4818-b363-4c1afae55289 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:02.622859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7f296-2245-4818-b363-4c1afae55289 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:02.622859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7f296-2245-4818-b363-4c1afae55289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:02.632654 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7f296-2245-4818-b363-4c1afae55289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:02.634085 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7f296-2245-4818-b363-4c1afae55289 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:02.643892 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 769/1533] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:06.509679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3e632-f0a6-46a2-8563-7cb07ca6d55f None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:06.510221 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3e632-f0a6-46a2-8563-7cb07ca6d55f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:06.510221 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3e632-f0a6-46a2-8563-7cb07ca6d55f None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:06.511737 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:58:06.517533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86607ffd-9e5b-45c7-a88b-c57c281b0a9f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:06.517960 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86607ffd-9e5b-45c7-a88b-c57c281b0a9f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:06.518067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86607ffd-9e5b-45c7-a88b-c57c281b0a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:06.526413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=74 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:06.541501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-86607ffd-9e5b-45c7-a88b-c57c281b0a9f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:58:06.542767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-86607ffd-9e5b-45c7-a88b-c57c281b0a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:06.552175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:58:06.857070 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3e632-f0a6-46a2-8563-7cb07ca6d55f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:06.858754 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-cba3e632-f0a6-46a2-8563-7cb07ca6d55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:06.916749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3e632-f0a6-46a2-8563-7cb07ca6d55f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:06.920414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4d626-7eba-4f73-8c0d-f5ce5faac7b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:06.921058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4d626-7eba-4f73-8c0d-f5ce5faac7b9 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:06.921296 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4d626-7eba-4f73-8c0d-f5ce5faac7b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:06.921562 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4d626-7eba-4f73-8c0d-f5ce5faac7b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:06.931612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4d626-7eba-4f73-8c0d-f5ce5faac7b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:06.933083 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4d626-7eba-4f73-8c0d-f5ce5faac7b9 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:06.942801 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 766/1536] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:09.149877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401a978-3583-46e7-8ea8-0a68c159f743 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:09.150448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401a978-3583-46e7-8ea8-0a68c159f743 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:09.150448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401a978-3583-46e7-8ea8-0a68c159f743 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:09.152099 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:58:09.157942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b27bb-1d5b-48d4-b638-4c46698ecf16 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:09.158463 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b27bb-1d5b-48d4-b638-4c46698ecf16 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:09.158578 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b27bb-1d5b-48d4-b638-4c46698ecf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:09.178783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-5f5b27bb-1d5b-48d4-b638-4c46698ecf16 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:58:09.180018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5f5b27bb-1d5b-48d4-b638-4c46698ecf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:09.187933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:58:09.491243 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401a978-3583-46e7-8ea8-0a68c159f743 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:09.492904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6401a978-3583-46e7-8ea8-0a68c159f743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:09.506655 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401a978-3583-46e7-8ea8-0a68c159f743 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:09.510178 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413125f-2f3d-4e28-b3da-a9b6da4d08aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:09.510804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413125f-2f3d-4e28-b3da-a9b6da4d08aa service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:09.511107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413125f-2f3d-4e28-b3da-a9b6da4d08aa service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:09.511377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413125f-2f3d-4e28-b3da-a9b6da4d08aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:09.521452 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413125f-2f3d-4e28-b3da-a9b6da4d08aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:09.522872 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413125f-2f3d-4e28-b3da-a9b6da4d08aa service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:09.532686 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 772/1539] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:19.173087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=54 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:19.528470 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=8 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:34.562408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff516bfb-3de0-4323-b994-b75ee6bcf9e9 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:34.563072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff516bfb-3de0-4323-b994-b75ee6bcf9e9 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:34.563072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff516bfb-3de0-4323-b994-b75ee6bcf9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:34.598010 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ff516bfb-3de0-4323-b994-b75ee6bcf9e9 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:34.599523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ff516bfb-3de0-4323-b994-b75ee6bcf9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:34.613264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 768/1540] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 08 13:59:34.623406 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548725f-d814-41fb-a5e4-6b948ff96b54 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:34.623996 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548725f-d814-41fb-a5e4-6b948ff96b54 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:34.623996 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548725f-d814-41fb-a5e4-6b948ff96b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:34.649901 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a548725f-d814-41fb-a5e4-6b948ff96b54 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:34.651506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a548725f-d814-41fb-a5e4-6b948ff96b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:34.663602 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 773/1541] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:34.668637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff516bfb-3de0-4323-b994-b75ee6bcf9e9 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:34.684280 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff516bfb-3de0-4323-b994-b75ee6bcf9e9 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:34.692645 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7d361-c933-4472-9139-d495386a784b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:34.693292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7d361-c933-4472-9139-d495386a784b admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:34.693575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7d361-c933-4472-9139-d495386a784b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:34.693851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7d361-c933-4472-9139-d495386a784b admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:34.694530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7d361-c933-4472-9139-d495386a784b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:59:34.694837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7d361-c933-4472-9139-d495386a784b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:34.696663 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7d361-c933-4472-9139-d495386a784b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:34.712595 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 769/1542] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 13:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:59:34.728846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548725f-d814-41fb-a5e4-6b948ff96b54 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:34.750443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548725f-d814-41fb-a5e4-6b948ff96b54 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:34.762291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8748bac-1e14-4bd7-a856-31185d150e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:34.766975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8748bac-1e14-4bd7-a856-31185d150e26 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:34.766975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8748bac-1e14-4bd7-a856-31185d150e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:34.766975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8748bac-1e14-4bd7-a856-31185d150e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:34.766975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8748bac-1e14-4bd7-a856-31185d150e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:34.766975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8748bac-1e14-4bd7-a856-31185d150e26 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:34.942175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 774/1543] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:34 2022] POST /identity/v3/projects => generated 371 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:34.947079 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7d361-c933-4472-9139-d495386a784b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:34.963891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7d361-c933-4472-9139-d495386a784b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:34.966596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fc8d8-dffd-4559-8dfe-e04eaa85abac None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:34.966883 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fc8d8-dffd-4559-8dfe-e04eaa85abac None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:34.967197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fc8d8-dffd-4559-8dfe-e04eaa85abac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.000348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-fe5fc8d8-dffd-4559-8dfe-e04eaa85abac None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:35.001636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fe5fc8d8-dffd-4559-8dfe-e04eaa85abac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:35.066025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e483259-2685-4033-aba9-16ea4032b920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.067087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e483259-2685-4033-aba9-16ea4032b920 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.067608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e483259-2685-4033-aba9-16ea4032b920 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.068123 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e483259-2685-4033-aba9-16ea4032b920 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.069041 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e483259-2685-4033-aba9-16ea4032b920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.071696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e483259-2685-4033-aba9-16ea4032b920 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.120020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 775/1544] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:59:35.129336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143f9eb-7c2a-4867-a6b1-f4af8fb6111d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.129652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143f9eb-7c2a-4867-a6b1-f4af8fb6111d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.129892 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143f9eb-7c2a-4867-a6b1-f4af8fb6111d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.168453 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-8143f9eb-7c2a-4867-a6b1-f4af8fb6111d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:35.170102 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8143f9eb-7c2a-4867-a6b1-f4af8fb6111d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:35.187222 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 776/1545] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:59:35.189017 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311baa09-44ff-4290-ae64-bb4e85cff2ac None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.189412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311baa09-44ff-4290-ae64-bb4e85cff2ac None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.189727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311baa09-44ff-4290-ae64-bb4e85cff2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.193996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 770/1546] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:34 2022] POST /identity/v3/users => generated 286 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:35.195969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6257965-bc09-454f-ab2c-932ba16068a1 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.196602 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6257965-bc09-454f-ab2c-932ba16068a1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.197319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6257965-bc09-454f-ab2c-932ba16068a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.235540 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d6257965-bc09-454f-ab2c-932ba16068a1 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:35.235966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-311baa09-44ff-4290-ae64-bb4e85cff2ac None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:35.237246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d6257965-bc09-454f-ab2c-932ba16068a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:35.237545 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-311baa09-44ff-4290-ae64-bb4e85cff2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:35.246682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 777/1547] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:35.247613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311baa09-44ff-4290-ae64-bb4e85cff2ac None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.261027 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 771/1548] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:35.264001 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce379f7f-ab9f-4a4e-aea8-9e0593ce7472 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.264386 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce379f7f-ab9f-4a4e-aea8-9e0593ce7472 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.264801 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce379f7f-ab9f-4a4e-aea8-9e0593ce7472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.266096 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311baa09-44ff-4290-ae64-bb4e85cff2ac None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.290632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0fb5a-520e-4dda-995a-acb53d49ee2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.291314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0fb5a-520e-4dda-995a-acb53d49ee2f admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.291595 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0fb5a-520e-4dda-995a-acb53d49ee2f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.291864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0fb5a-520e-4dda-995a-acb53d49ee2f admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.296649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0fb5a-520e-4dda-995a-acb53d49ee2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.298483 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0fb5a-520e-4dda-995a-acb53d49ee2f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.307937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ce379f7f-ab9f-4a4e-aea8-9e0593ce7472 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:35.310221 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ce379f7f-ab9f-4a4e-aea8-9e0593ce7472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:35.311877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 778/1549] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:35.313424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c0adc-5f15-4ba2-8cd2-800d52351e7a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.313704 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c0adc-5f15-4ba2-8cd2-800d52351e7a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.313973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c0adc-5f15-4ba2-8cd2-800d52351e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.318929 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 772/1550] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:35.320261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b9178-6c43-4a8e-ad5a-40518847c73b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.320549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b9178-6c43-4a8e-ad5a-40518847c73b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.320817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b9178-6c43-4a8e-ad5a-40518847c73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.340953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-fb2c0adc-5f15-4ba2-8cd2-800d52351e7a None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:35.342373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fb2c0adc-5f15-4ba2-8cd2-800d52351e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:35.346018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c84b9178-6c43-4a8e-ad5a-40518847c73b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:35.347216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c84b9178-6c43-4a8e-ad5a-40518847c73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:35.354960 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 779/1551] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:35.358945 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 773/1552] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:35.359140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c0adc-5f15-4ba2-8cd2-800d52351e7a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.359292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f90f8-2f14-46bc-988f-cb7f1c5d0f5b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.359292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f90f8-2f14-46bc-988f-cb7f1c5d0f5b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.359292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f90f8-2f14-46bc-988f-cb7f1c5d0f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.369947 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c0adc-5f15-4ba2-8cd2-800d52351e7a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.375141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049a2f6-5b5a-48b9-aabf-2e4114e74510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.375758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049a2f6-5b5a-48b9-aabf-2e4114e74510 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.376036 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049a2f6-5b5a-48b9-aabf-2e4114e74510 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.376305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049a2f6-5b5a-48b9-aabf-2e4114e74510 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.376948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049a2f6-5b5a-48b9-aabf-2e4114e74510 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:59:35.377246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049a2f6-5b5a-48b9-aabf-2e4114e74510 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.378288 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: WARNING py.warnings [None req-8049a2f6-5b5a-48b9-aabf-2e4114e74510 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:59:35.378288 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: warnings.warn(msg) Sep 08 13:59:35.378288 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: Sep 08 13:59:35.379201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049a2f6-5b5a-48b9-aabf-2e4114e74510 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.384731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-691f90f8-2f14-46bc-988f-cb7f1c5d0f5b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:35.385520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-691f90f8-2f14-46bc-988f-cb7f1c5d0f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:35.393284 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 780/1553] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 13:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:59:35.395830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d2d8b-7c43-4b76-9cf4-0fe54ea36e9f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.396660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d2d8b-7c43-4b76-9cf4-0fe54ea36e9f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.396934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d2d8b-7c43-4b76-9cf4-0fe54ea36e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.402343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 774/1554] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:35.403250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f90f8-2f14-46bc-988f-cb7f1c5d0f5b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.415970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f90f8-2f14-46bc-988f-cb7f1c5d0f5b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.421599 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e846dc2-59ef-4c50-8e3c-6209b3dbd263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.422186 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e846dc2-59ef-4c50-8e3c-6209b3dbd263 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.422401 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e846dc2-59ef-4c50-8e3c-6209b3dbd263 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.422685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e846dc2-59ef-4c50-8e3c-6209b3dbd263 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.422879 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-0e1d2d8b-7c43-4b76-9cf4-0fe54ea36e9f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:35.423341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e846dc2-59ef-4c50-8e3c-6209b3dbd263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.424169 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0e1d2d8b-7c43-4b76-9cf4-0fe54ea36e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:35.424956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e846dc2-59ef-4c50-8e3c-6209b3dbd263 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.435025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 781/1555] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:35.436224 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d2d8b-7c43-4b76-9cf4-0fe54ea36e9f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.440403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 775/1556] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:35.440403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3582d-0264-4ee2-958f-09b356c9c64c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.440403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3582d-0264-4ee2-958f-09b356c9c64c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.440403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3582d-0264-4ee2-958f-09b356c9c64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.447617 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d2d8b-7c43-4b76-9cf4-0fe54ea36e9f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.452858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5d904-2f5a-4341-be4d-9f18c0c56f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.453584 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5d904-2f5a-4341-be4d-9f18c0c56f73 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.453860 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5d904-2f5a-4341-be4d-9f18c0c56f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.454154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5d904-2f5a-4341-be4d-9f18c0c56f73 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.455037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5d904-2f5a-4341-be4d-9f18c0c56f73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:59:35.455469 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5d904-2f5a-4341-be4d-9f18c0c56f73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.457401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5d904-2f5a-4341-be4d-9f18c0c56f73 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.468209 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9de3582d-0264-4ee2-958f-09b356c9c64c None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:35.469205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9de3582d-0264-4ee2-958f-09b356c9c64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:35.478144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 782/1557] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 13:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:59:35.478144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa80029-6eb1-49dd-a49e-7a9f0a6622fa None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.478144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa80029-6eb1-49dd-a49e-7a9f0a6622fa None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.478144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa80029-6eb1-49dd-a49e-7a9f0a6622fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.491056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 776/1558] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:35.494225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09589511-a7f3-449e-818f-4d2f0a1c6069 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.494225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09589511-a7f3-449e-818f-4d2f0a1c6069 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.494225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09589511-a7f3-449e-818f-4d2f0a1c6069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.502182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1fa80029-6eb1-49dd-a49e-7a9f0a6622fa None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:35.503477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1fa80029-6eb1-49dd-a49e-7a9f0a6622fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:35.511796 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 783/1559] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:35.512514 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa80029-6eb1-49dd-a49e-7a9f0a6622fa None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.515785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-09589511-a7f3-449e-818f-4d2f0a1c6069 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:35.517334 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-09589511-a7f3-449e-818f-4d2f0a1c6069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:35.522181 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa80029-6eb1-49dd-a49e-7a9f0a6622fa None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.525735 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 777/1560] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:35.526640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09589511-a7f3-449e-818f-4d2f0a1c6069 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.530834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd268c-6b68-425c-9f6f-26b94ef8c716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.531493 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd268c-6b68-425c-9f6f-26b94ef8c716 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.531747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd268c-6b68-425c-9f6f-26b94ef8c716 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.531998 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd268c-6b68-425c-9f6f-26b94ef8c716 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.532659 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd268c-6b68-425c-9f6f-26b94ef8c716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.535024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd268c-6b68-425c-9f6f-26b94ef8c716 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.540425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09589511-a7f3-449e-818f-4d2f0a1c6069 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.550637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313e8db-a50f-42b1-9d10-84eb111dab66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.551452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313e8db-a50f-42b1-9d10-84eb111dab66 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.551861 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313e8db-a50f-42b1-9d10-84eb111dab66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.552273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313e8db-a50f-42b1-9d10-84eb111dab66 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.553034 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313e8db-a50f-42b1-9d10-84eb111dab66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:59:35.553462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313e8db-a50f-42b1-9d10-84eb111dab66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.555454 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313e8db-a50f-42b1-9d10-84eb111dab66 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.569373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 778/1561] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 13:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:59:35.570330 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313e8db-a50f-42b1-9d10-84eb111dab66 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.580392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313e8db-a50f-42b1-9d10-84eb111dab66 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.585075 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0dc71-34b9-43c0-8738-5fbbbf50ee83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.585715 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0dc71-34b9-43c0-8738-5fbbbf50ee83 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.585996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0dc71-34b9-43c0-8738-5fbbbf50ee83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.586271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0dc71-34b9-43c0-8738-5fbbbf50ee83 admin admin] PATH_INFO: `/v3/projects/266ca24a1fb44f9992e9eb372695bdc6/users/73601a32878240dd85194e733a671a86/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.603221 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0dc71-34b9-43c0-8738-5fbbbf50ee83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=266ca24a1fb44f9992e9eb372695bdc6, user_id=73601a32878240dd85194e733a671a86, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.604337 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: WARNING py.warnings [None req-a3d0dc71-34b9-43c0-8738-5fbbbf50ee83 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:59:35.604337 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: warnings.warn(msg) Sep 08 13:59:35.604337 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: Sep 08 13:59:35.606790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0dc71-34b9-43c0-8738-5fbbbf50ee83 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.643751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 784/1562] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:35.643751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c142057-328d-4f89-a2d3-5777ebbf82d2 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.643751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c142057-328d-4f89-a2d3-5777ebbf82d2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.643751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c142057-328d-4f89-a2d3-5777ebbf82d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.662754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-7c142057-328d-4f89-a2d3-5777ebbf82d2 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:35.664014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7c142057-328d-4f89-a2d3-5777ebbf82d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:35.693589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:35.710030 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 779/1563] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:35 2022] PUT /identity/v3/projects/266ca24a1fb44f9992e9eb372695bdc6/users/73601a32878240dd85194e733a671a86/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:35.710859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0dc71-34b9-43c0-8738-5fbbbf50ee83 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.742269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 785/1564] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:35.742269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfccaca-388d-494f-bca1-e5b293a27239 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.742269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfccaca-388d-494f-bca1-e5b293a27239 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.742269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfccaca-388d-494f-bca1-e5b293a27239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.761117 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-edfccaca-388d-494f-bca1-e5b293a27239 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:35.762448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-edfccaca-388d-494f-bca1-e5b293a27239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:35.770107 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:35.789399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0dc71-34b9-43c0-8738-5fbbbf50ee83 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.795716 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8fad0-f468-4ba1-a7c1-0a83ffeddeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.796411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8fad0-f468-4ba1-a7c1-0a83ffeddeb2 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.796680 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8fad0-f468-4ba1-a7c1-0a83ffeddeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.796956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8fad0-f468-4ba1-a7c1-0a83ffeddeb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.797704 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8fad0-f468-4ba1-a7c1-0a83ffeddeb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.798853 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: WARNING py.warnings [None req-c2e8fad0-f468-4ba1-a7c1-0a83ffeddeb2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:59:35.798853 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: warnings.warn(msg) Sep 08 13:59:35.798853 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: Sep 08 13:59:35.800278 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8fad0-f468-4ba1-a7c1-0a83ffeddeb2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.836296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 786/1565] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:35.836296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfccaca-388d-494f-bca1-e5b293a27239 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.849994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfccaca-388d-494f-bca1-e5b293a27239 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.855919 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e93bc9-4573-48d4-ab7a-0a730fc5c838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.856602 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e93bc9-4573-48d4-ab7a-0a730fc5c838 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.856904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e93bc9-4573-48d4-ab7a-0a730fc5c838 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.857176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e93bc9-4573-48d4-ab7a-0a730fc5c838 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.857839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e93bc9-4573-48d4-ab7a-0a730fc5c838 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:59:35.858165 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e93bc9-4573-48d4-ab7a-0a730fc5c838 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.859992 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e93bc9-4573-48d4-ab7a-0a730fc5c838 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.873063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 787/1566] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 13:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:59:35.874026 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e93bc9-4573-48d4-ab7a-0a730fc5c838 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.886705 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e93bc9-4573-48d4-ab7a-0a730fc5c838 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.892722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd45f3e-b398-483c-9ae5-06a8c6515192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.893382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd45f3e-b398-483c-9ae5-06a8c6515192 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.895574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd45f3e-b398-483c-9ae5-06a8c6515192 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.895574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd45f3e-b398-483c-9ae5-06a8c6515192 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.895574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd45f3e-b398-483c-9ae5-06a8c6515192 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:59:35.895574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd45f3e-b398-483c-9ae5-06a8c6515192 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.903641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd45f3e-b398-483c-9ae5-06a8c6515192 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.903826 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 780/1567] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/projects => generated 405 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:35.903826 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8fad0-f468-4ba1-a7c1-0a83ffeddeb2 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.914157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 788/1568] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 13:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:59:35.915336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd45f3e-b398-483c-9ae5-06a8c6515192 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.931351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd45f3e-b398-483c-9ae5-06a8c6515192 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.976459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8fad0-f468-4ba1-a7c1-0a83ffeddeb2 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.982831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1b2b5-887a-4093-9cfc-b7d69fe29506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.983491 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1b2b5-887a-4093-9cfc-b7d69fe29506 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.983764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1b2b5-887a-4093-9cfc-b7d69fe29506 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.984034 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1b2b5-887a-4093-9cfc-b7d69fe29506 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.984660 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1b2b5-887a-4093-9cfc-b7d69fe29506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.986575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1b2b5-887a-4093-9cfc-b7d69fe29506 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.993130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e01ba5-e509-41b3-9d14-dc7c5e48919f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.993742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e01ba5-e509-41b3-9d14-dc7c5e48919f admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.994021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e01ba5-e509-41b3-9d14-dc7c5e48919f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.994299 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e01ba5-e509-41b3-9d14-dc7c5e48919f admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.994951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e01ba5-e509-41b3-9d14-dc7c5e48919f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.995979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: WARNING py.warnings [None req-88e01ba5-e509-41b3-9d14-dc7c5e48919f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:59:35.995979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: warnings.warn(msg) Sep 08 13:59:35.995979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: Sep 08 13:59:35.997182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e01ba5-e509-41b3-9d14-dc7c5e48919f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:36.094464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 781/1569] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/projects => generated 391 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:36.094464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1342e-d459-44ab-ad5f-c60c1b99327f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.094464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1342e-d459-44ab-ad5f-c60c1b99327f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.094464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1342e-d459-44ab-ad5f-c60c1b99327f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.114247 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 789/1570] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:36.114247 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e01ba5-e509-41b3-9d14-dc7c5e48919f admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:36.122832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9ee1342e-d459-44ab-ad5f-c60c1b99327f None None] MFA Rules not processed for user `73601a32878240dd85194e733a671a86`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:36.124075 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9ee1342e-d459-44ab-ad5f-c60c1b99327f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:36.127254 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e01ba5-e509-41b3-9d14-dc7c5e48919f admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:36.163786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:36.188306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f8ab2-03d2-4b67-989c-f58f518259ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:36.188919 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f8ab2-03d2-4b67-989c-f58f518259ef admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.189228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f8ab2-03d2-4b67-989c-f58f518259ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.189501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f8ab2-03d2-4b67-989c-f58f518259ef admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.190243 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f8ab2-03d2-4b67-989c-f58f518259ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:59:36.190550 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f8ab2-03d2-4b67-989c-f58f518259ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:36.192294 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f8ab2-03d2-4b67-989c-f58f518259ef admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:36.197240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 782/1571] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:36.198099 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1342e-d459-44ab-ad5f-c60c1b99327f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:36.206683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 790/1572] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 13:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:59:36.207623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f8ab2-03d2-4b67-989c-f58f518259ef admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:36.220117 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1342e-d459-44ab-ad5f-c60c1b99327f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:36.225228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f8ab2-03d2-4b67-989c-f58f518259ef admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:36.225880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b4d42-4d7e-4802-b804-92deab0536d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:36.226669 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b4d42-4d7e-4802-b804-92deab0536d2 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.227056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b4d42-4d7e-4802-b804-92deab0536d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.227442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b4d42-4d7e-4802-b804-92deab0536d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.228100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b4d42-4d7e-4802-b804-92deab0536d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:59:36.228632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b4d42-4d7e-4802-b804-92deab0536d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:36.230222 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b4d42-4d7e-4802-b804-92deab0536d2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:36.230925 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e46c36-2ee4-4a43-9253-4c14c3152bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:36.231540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e46c36-2ee4-4a43-9253-4c14c3152bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.231934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e46c36-2ee4-4a43-9253-4c14c3152bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.232102 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e46c36-2ee4-4a43-9253-4c14c3152bb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.232895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e46c36-2ee4-4a43-9253-4c14c3152bb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:36.235015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e46c36-2ee4-4a43-9253-4c14c3152bb1 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:36.243138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 783/1573] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 13:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:59:36.244100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b4d42-4d7e-4802-b804-92deab0536d2 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:36.259016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b4d42-4d7e-4802-b804-92deab0536d2 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:36.264723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead37f86-39cb-4c3b-a473-2b41044f9f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:36.265408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead37f86-39cb-4c3b-a473-2b41044f9f1f admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.265782 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead37f86-39cb-4c3b-a473-2b41044f9f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.266148 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead37f86-39cb-4c3b-a473-2b41044f9f1f admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.266867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead37f86-39cb-4c3b-a473-2b41044f9f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:36.268900 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead37f86-39cb-4c3b-a473-2b41044f9f1f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:36.359225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 791/1574] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:36 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:36.360139 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e46c36-2ee4-4a43-9253-4c14c3152bb1 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:36.387880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 784/1575] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:36 2022] POST /identity/v3/users => generated 303 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:36.393539 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead37f86-39cb-4c3b-a473-2b41044f9f1f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:36.399231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead37f86-39cb-4c3b-a473-2b41044f9f1f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:36.409237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:36.434628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e46c36-2ee4-4a43-9253-4c14c3152bb1 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:36.439442 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c56c52-eb60-4545-a45e-313d9dfc1b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:36.440030 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c56c52-eb60-4545-a45e-313d9dfc1b0f admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.440303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c56c52-eb60-4545-a45e-313d9dfc1b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.440571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c56c52-eb60-4545-a45e-313d9dfc1b0f admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.441191 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c56c52-eb60-4545-a45e-313d9dfc1b0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:36.443118 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c56c52-eb60-4545-a45e-313d9dfc1b0f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:36.467678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e9013-1498-4abf-88a5-1fc0ffd9d01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:36.468281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e9013-1498-4abf-88a5-1fc0ffd9d01a admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.468553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e9013-1498-4abf-88a5-1fc0ffd9d01a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.468820 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e9013-1498-4abf-88a5-1fc0ffd9d01a admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.469446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e9013-1498-4abf-88a5-1fc0ffd9d01a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:36.471448 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e9013-1498-4abf-88a5-1fc0ffd9d01a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:36.548387 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 792/1576] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:36 2022] POST /identity/v3/projects => generated 371 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:36.549309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c56c52-eb60-4545-a45e-313d9dfc1b0f admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:36.577690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 785/1577] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:36 2022] POST /identity/v3/users => generated 296 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:36.580789 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e9013-1498-4abf-88a5-1fc0ffd9d01a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:36.592686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e9013-1498-4abf-88a5-1fc0ffd9d01a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:36.620086 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c56c52-eb60-4545-a45e-313d9dfc1b0f admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:36.625308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55989d9-fcf4-44d2-8d53-f173ea20edac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:36.625938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55989d9-fcf4-44d2-8d53-f173ea20edac admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.629747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55989d9-fcf4-44d2-8d53-f173ea20edac admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.629747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55989d9-fcf4-44d2-8d53-f173ea20edac admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.629747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55989d9-fcf4-44d2-8d53-f173ea20edac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:36.629747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55989d9-fcf4-44d2-8d53-f173ea20edac admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:36.640891 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 793/1578] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:36.642342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abdf14-a5fa-475c-b110-bba1e7181aa0 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.642617 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abdf14-a5fa-475c-b110-bba1e7181aa0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.642921 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abdf14-a5fa-475c-b110-bba1e7181aa0 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.644563 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 794/1579] 10.222.0.91 () {54 vars in 962 bytes} [Thu Sep 8 13:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:59:36.645263 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abdf14-a5fa-475c-b110-bba1e7181aa0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:36.658840 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abdf14-a5fa-475c-b110-bba1e7181aa0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:36.664783 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33879e43-d566-4d52-b239-ce482a658095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:36.665522 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33879e43-d566-4d52-b239-ce482a658095 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.665992 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33879e43-d566-4d52-b239-ce482a658095 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.666376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33879e43-d566-4d52-b239-ce482a658095 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.667275 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33879e43-d566-4d52-b239-ce482a658095 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:36.668194 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba6e50-aaa8-4b33-8df8-a6c02706fce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:36.668800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba6e50-aaa8-4b33-8df8-a6c02706fce8 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.669075 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba6e50-aaa8-4b33-8df8-a6c02706fce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.669368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba6e50-aaa8-4b33-8df8-a6c02706fce8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.669475 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33879e43-d566-4d52-b239-ce482a658095 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:36.669989 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba6e50-aaa8-4b33-8df8-a6c02706fce8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:36.671940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba6e50-aaa8-4b33-8df8-a6c02706fce8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:36.777008 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 795/1580] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:36 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:36.778082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33879e43-d566-4d52-b239-ce482a658095 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:36.787793 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:36.806953 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 786/1581] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:36 2022] POST /identity/v3/projects => generated 403 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:36.810626 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba6e50-aaa8-4b33-8df8-a6c02706fce8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:36.853896 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33879e43-d566-4d52-b239-ce482a658095 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:36.881507 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba6e50-aaa8-4b33-8df8-a6c02706fce8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:36.886860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28683220-efcf-45aa-a437-66f4a4a7663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:36.887499 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28683220-efcf-45aa-a437-66f4a4a7663b admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.887773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28683220-efcf-45aa-a437-66f4a4a7663b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.888064 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28683220-efcf-45aa-a437-66f4a4a7663b admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.888701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28683220-efcf-45aa-a437-66f4a4a7663b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:36.890367 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28683220-efcf-45aa-a437-66f4a4a7663b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:36.901762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 787/1582] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:36.902769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28683220-efcf-45aa-a437-66f4a4a7663b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:36.912960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28683220-efcf-45aa-a437-66f4a4a7663b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:36.918405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b928f-6067-45c3-aa1a-9877db6efab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:36.918783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb1ffb-6bb3-413f-ba3f-d910e0bc1bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:36.919169 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b928f-6067-45c3-aa1a-9877db6efab0 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.919315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b928f-6067-45c3-aa1a-9877db6efab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.919570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb1ffb-6bb3-413f-ba3f-d910e0bc1bff admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.919697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b928f-6067-45c3-aa1a-9877db6efab0 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.919797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb1ffb-6bb3-413f-ba3f-d910e0bc1bff admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.919920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb1ffb-6bb3-413f-ba3f-d910e0bc1bff admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.920265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b928f-6067-45c3-aa1a-9877db6efab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:36.920558 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb1ffb-6bb3-413f-ba3f-d910e0bc1bff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:36.922202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b928f-6067-45c3-aa1a-9877db6efab0 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:36.922570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb1ffb-6bb3-413f-ba3f-d910e0bc1bff admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.009395 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 796/1583] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:36 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:37.010538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b928f-6067-45c3-aa1a-9877db6efab0 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.023156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b928f-6067-45c3-aa1a-9877db6efab0 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.029927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b569522-09fe-4ee9-bcaa-e23b7cdb2620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.030569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b569522-09fe-4ee9-bcaa-e23b7cdb2620 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.031012 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b569522-09fe-4ee9-bcaa-e23b7cdb2620 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.034051 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b569522-09fe-4ee9-bcaa-e23b7cdb2620 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.034051 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b569522-09fe-4ee9-bcaa-e23b7cdb2620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.034372 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 788/1584] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:36 2022] POST /identity/v3/users => generated 286 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:37.034372 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb1ffb-6bb3-413f-ba3f-d910e0bc1bff admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.034638 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b569522-09fe-4ee9-bcaa-e23b7cdb2620 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.043427 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:37.046053 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 797/1585] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:37.046987 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba9b3c-6fd6-48f8-b41e-06cbd943826b None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.047279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba9b3c-6fd6-48f8-b41e-06cbd943826b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.047590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba9b3c-6fd6-48f8-b41e-06cbd943826b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.047944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb1ffb-6bb3-413f-ba3f-d910e0bc1bff admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.053087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a99df1-bb71-4713-940c-6569a26598a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.053718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a99df1-bb71-4713-940c-6569a26598a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.054001 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a99df1-bb71-4713-940c-6569a26598a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.054293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a99df1-bb71-4713-940c-6569a26598a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.054959 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a99df1-bb71-4713-940c-6569a26598a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.056609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a99df1-bb71-4713-940c-6569a26598a8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.068516 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 789/1586] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:37.069471 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a99df1-bb71-4713-940c-6569a26598a8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.080071 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a99df1-bb71-4713-940c-6569a26598a8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.086295 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c809b7a-3e28-4599-922f-43ace80f988e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.087009 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c809b7a-3e28-4599-922f-43ace80f988e admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.087288 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c809b7a-3e28-4599-922f-43ace80f988e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.087597 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c809b7a-3e28-4599-922f-43ace80f988e admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.090127 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c809b7a-3e28-4599-922f-43ace80f988e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.090316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c809b7a-3e28-4599-922f-43ace80f988e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.094532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-08ba9b3c-6fd6-48f8-b41e-06cbd943826b None None] MFA Rules not processed for user `38127f41852d412ea6ad270f1618fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:37.096057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-08ba9b3c-6fd6-48f8-b41e-06cbd943826b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:37.168425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 798/1587] 10.222.0.91 () {58 vars in 1059 bytes} [Thu Sep 8 13:59:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:37.169337 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba9b3c-6fd6-48f8-b41e-06cbd943826b None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.169549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 790/1588] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:37 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:37.173537 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c809b7a-3e28-4599-922f-43ace80f988e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.180415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba9b3c-6fd6-48f8-b41e-06cbd943826b None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.187072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0584ff-5db8-4d8a-b000-81a13c599f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.187279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c809b7a-3e28-4599-922f-43ace80f988e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.187391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0584ff-5db8-4d8a-b000-81a13c599f12 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.187391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0584ff-5db8-4d8a-b000-81a13c599f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.187391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0584ff-5db8-4d8a-b000-81a13c599f12 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.187878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0584ff-5db8-4d8a-b000-81a13c599f12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.189801 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0584ff-5db8-4d8a-b000-81a13c599f12 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.191039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c62f6-7d39-42de-a26d-3423f13b1288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.193328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c62f6-7d39-42de-a26d-3423f13b1288 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.193328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c62f6-7d39-42de-a26d-3423f13b1288 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.193328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c62f6-7d39-42de-a26d-3423f13b1288 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.193328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c62f6-7d39-42de-a26d-3423f13b1288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.194454 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c62f6-7d39-42de-a26d-3423f13b1288 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.200888 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:37.205547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 791/1589] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:37.206408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c62f6-7d39-42de-a26d-3423f13b1288 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.216764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c62f6-7d39-42de-a26d-3423f13b1288 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.221649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6852343-c70c-4497-830f-3f0c8b9b4574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.222316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6852343-c70c-4497-830f-3f0c8b9b4574 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.222558 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6852343-c70c-4497-830f-3f0c8b9b4574 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.222829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6852343-c70c-4497-830f-3f0c8b9b4574 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.223497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6852343-c70c-4497-830f-3f0c8b9b4574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.225118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6852343-c70c-4497-830f-3f0c8b9b4574 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.237671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 792/1590] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:37.238472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6852343-c70c-4497-830f-3f0c8b9b4574 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.248909 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6852343-c70c-4497-830f-3f0c8b9b4574 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.253603 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf46d7-2edd-474c-ab8a-e2785b37ddba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.254175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf46d7-2edd-474c-ab8a-e2785b37ddba admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.254452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf46d7-2edd-474c-ab8a-e2785b37ddba admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.254722 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf46d7-2edd-474c-ab8a-e2785b37ddba admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.255373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf46d7-2edd-474c-ab8a-e2785b37ddba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.256997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf46d7-2edd-474c-ab8a-e2785b37ddba admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.267597 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 799/1591] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:37 2022] POST /identity/v3/users => generated 302 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:37.269065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 793/1592] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:37.270129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf46d7-2edd-474c-ab8a-e2785b37ddba admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.271980 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0584ff-5db8-4d8a-b000-81a13c599f12 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.281516 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf46d7-2edd-474c-ab8a-e2785b37ddba admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.284736 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0584ff-5db8-4d8a-b000-81a13c599f12 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.286056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab45e6e-a07b-4f0b-b862-59720f3fed33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.286700 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab45e6e-a07b-4f0b-b862-59720f3fed33 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.287093 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab45e6e-a07b-4f0b-b862-59720f3fed33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.287365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab45e6e-a07b-4f0b-b862-59720f3fed33 admin admin] PATH_INFO: `/v3/projects/f768b4d35c1b4713b58e3d582adcc93f/users/5771e9d0c5c94306b3ec571916acdcaa/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.290237 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa4ae8-1a45-4dcb-9de0-4a731e207c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.290978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4ae8-1a45-4dcb-9de0-4a731e207c60 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.291470 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4ae8-1a45-4dcb-9de0-4a731e207c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.291693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4ae8-1a45-4dcb-9de0-4a731e207c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.292384 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa4ae8-1a45-4dcb-9de0-4a731e207c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.295546 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa4ae8-1a45-4dcb-9de0-4a731e207c60 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.306524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 800/1593] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:37.307385 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa4ae8-1a45-4dcb-9de0-4a731e207c60 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.312642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab45e6e-a07b-4f0b-b862-59720f3fed33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f768b4d35c1b4713b58e3d582adcc93f, user_id=5771e9d0c5c94306b3ec571916acdcaa, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.315375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab45e6e-a07b-4f0b-b862-59720f3fed33 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.320571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa4ae8-1a45-4dcb-9de0-4a731e207c60 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.326122 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c9589-338d-46d2-90bd-c8bad3919059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.326788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c9589-338d-46d2-90bd-c8bad3919059 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.327276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c9589-338d-46d2-90bd-c8bad3919059 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.327632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c9589-338d-46d2-90bd-c8bad3919059 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.327980 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c9589-338d-46d2-90bd-c8bad3919059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.329653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c9589-338d-46d2-90bd-c8bad3919059 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.341043 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 801/1594] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:37.341886 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c9589-338d-46d2-90bd-c8bad3919059 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.354132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c9589-338d-46d2-90bd-c8bad3919059 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.360316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a73d8-0a5a-4805-879b-3ca824d7a42e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.360956 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a73d8-0a5a-4805-879b-3ca824d7a42e service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.361236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a73d8-0a5a-4805-879b-3ca824d7a42e service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.361562 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a73d8-0a5a-4805-879b-3ca824d7a42e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.408686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 794/1595] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:37 2022] PUT /identity/v3/projects/f768b4d35c1b4713b58e3d582adcc93f/users/5771e9d0c5c94306b3ec571916acdcaa/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:37.408686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab45e6e-a07b-4f0b-b862-59720f3fed33 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.425389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a73d8-0a5a-4805-879b-3ca824d7a42e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.426960 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a73d8-0a5a-4805-879b-3ca824d7a42e service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.436265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 802/1596] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:37.441035 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a73d8-0a5a-4805-879b-3ca824d7a42e service nova] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.447261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a73d8-0a5a-4805-879b-3ca824d7a42e service nova] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.492243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab45e6e-a07b-4f0b-b862-59720f3fed33 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.498463 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8fe1c-08be-4ee9-8ed0-5201ed97b682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.499371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8fe1c-08be-4ee9-8ed0-5201ed97b682 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.499749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8fe1c-08be-4ee9-8ed0-5201ed97b682 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.500093 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8fe1c-08be-4ee9-8ed0-5201ed97b682 admin admin] PATH_INFO: `/v3/projects/8ac10c5287324bfb80ed74e15efc5308/users/30b267a6fd414391a33389a481de607d/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.514506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57983a-1a6a-4265-9b97-e782e47e3093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.515259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57983a-1a6a-4265-9b97-e782e47e3093 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.515541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57983a-1a6a-4265-9b97-e782e47e3093 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.515753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57983a-1a6a-4265-9b97-e782e47e3093 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.516270 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57983a-1a6a-4265-9b97-e782e47e3093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.518085 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57983a-1a6a-4265-9b97-e782e47e3093 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.518482 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8fe1c-08be-4ee9-8ed0-5201ed97b682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac10c5287324bfb80ed74e15efc5308, user_id=30b267a6fd414391a33389a481de607d, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.521358 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8fe1c-08be-4ee9-8ed0-5201ed97b682 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.530421 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 803/1597] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:37.531405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57983a-1a6a-4265-9b97-e782e47e3093 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.541147 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57983a-1a6a-4265-9b97-e782e47e3093 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.546747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f78fe-4d0d-4876-8514-062da237a9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.547364 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f78fe-4d0d-4876-8514-062da237a9bf admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.547769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f78fe-4d0d-4876-8514-062da237a9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.548048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f78fe-4d0d-4876-8514-062da237a9bf admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.548668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f78fe-4d0d-4876-8514-062da237a9bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.550728 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f78fe-4d0d-4876-8514-062da237a9bf admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.561373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 804/1598] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:37.563952 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f78fe-4d0d-4876-8514-062da237a9bf admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.574391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f78fe-4d0d-4876-8514-062da237a9bf admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.579794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0048e-743e-4e3c-8e68-76927aed0105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.580397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0048e-743e-4e3c-8e68-76927aed0105 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.580678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0048e-743e-4e3c-8e68-76927aed0105 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.580949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0048e-743e-4e3c-8e68-76927aed0105 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.581593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0048e-743e-4e3c-8e68-76927aed0105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.583226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0048e-743e-4e3c-8e68-76927aed0105 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.593827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 795/1599] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:37 2022] PUT /identity/v3/projects/8ac10c5287324bfb80ed74e15efc5308/users/30b267a6fd414391a33389a481de607d/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:37.595819 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8fe1c-08be-4ee9-8ed0-5201ed97b682 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.601997 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 805/1600] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:37.603443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0048e-743e-4e3c-8e68-76927aed0105 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.616523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0048e-743e-4e3c-8e68-76927aed0105 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.669625 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:37.674458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8fe1c-08be-4ee9-8ed0-5201ed97b682 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.679413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd7cb8-d5d3-4ea9-81bc-6d31304de0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.679971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd7cb8-d5d3-4ea9-81bc-6d31304de0ac admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.680197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd7cb8-d5d3-4ea9-81bc-6d31304de0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.680476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd7cb8-d5d3-4ea9-81bc-6d31304de0ac admin admin] PATH_INFO: `/v3/projects/2078beb45799497ea367c552b6d8bcf9/users/e4916206c2ff42669276e0a93d6483ab/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.685196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:37.687704 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46e256-fa86-4808-a295-88efe42b1592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.688320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46e256-fa86-4808-a295-88efe42b1592 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.688605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46e256-fa86-4808-a295-88efe42b1592 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.688874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46e256-fa86-4808-a295-88efe42b1592 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.689518 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46e256-fa86-4808-a295-88efe42b1592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.691149 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46e256-fa86-4808-a295-88efe42b1592 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.699350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd7cb8-d5d3-4ea9-81bc-6d31304de0ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=2078beb45799497ea367c552b6d8bcf9, user_id=e4916206c2ff42669276e0a93d6483ab, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.702131 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd7cb8-d5d3-4ea9-81bc-6d31304de0ac admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.707234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 806/1601] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:37.708993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c54e4c-c62c-4151-871e-ca5e25782d19 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.712016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c54e4c-c62c-4151-871e-ca5e25782d19 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.712016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c54e4c-c62c-4151-871e-ca5e25782d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.730432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-00c54e4c-c62c-4151-871e-ca5e25782d19 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:37.733965 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-00c54e4c-c62c-4151-871e-ca5e25782d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:37.739753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 807/1602] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:37.740531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c54e4c-c62c-4151-871e-ca5e25782d19 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.751516 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c54e4c-c62c-4151-871e-ca5e25782d19 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.756456 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e6b70-6546-4b87-b234-dbf611ade1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.757132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e6b70-6546-4b87-b234-dbf611ade1a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.762510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e6b70-6546-4b87-b234-dbf611ade1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.762510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e6b70-6546-4b87-b234-dbf611ade1a6 admin admin] PATH_INFO: `/v3/projects/4d0ed4e4f54c47369721e871f0dec518/users/7b403bdc9cad4783b72bbf174199363c/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.773915 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e6b70-6546-4b87-b234-dbf611ade1a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0ed4e4f54c47369721e871f0dec518, user_id=7b403bdc9cad4783b72bbf174199363c, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.778255 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e6b70-6546-4b87-b234-dbf611ade1a6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.822683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 796/1603] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:37 2022] PUT /identity/v3/projects/2078beb45799497ea367c552b6d8bcf9/users/e4916206c2ff42669276e0a93d6483ab/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:37.823346 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd7cb8-d5d3-4ea9-81bc-6d31304de0ac admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.878069 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 808/1604] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:37 2022] PUT /identity/v3/projects/4d0ed4e4f54c47369721e871f0dec518/users/7b403bdc9cad4783b72bbf174199363c/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:37.879552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd53f1-6832-48b6-8c67-5728ea727d8d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.879861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd53f1-6832-48b6-8c67-5728ea727d8d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.880178 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd53f1-6832-48b6-8c67-5728ea727d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.891008 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd7cb8-d5d3-4ea9-81bc-6d31304de0ac admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.912431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-02bd53f1-6832-48b6-8c67-5728ea727d8d None None] MFA Rules not processed for user `30b267a6fd414391a33389a481de607d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:37.913706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-02bd53f1-6832-48b6-8c67-5728ea727d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:37.956795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab692e-7244-4672-a19c-805187df59d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.957426 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab692e-7244-4672-a19c-805187df59d2 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.957714 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab692e-7244-4672-a19c-805187df59d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.957988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab692e-7244-4672-a19c-805187df59d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.958636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab692e-7244-4672-a19c-805187df59d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:37.960296 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab692e-7244-4672-a19c-805187df59d2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:37.975685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 797/1605] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:37.975685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab692e-7244-4672-a19c-805187df59d2 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.983318 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab692e-7244-4672-a19c-805187df59d2 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:37.985643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 809/1606] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:37 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:37.987307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81a26d-987e-4ddb-aeb6-bf04c55cfe5c None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.987529 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81a26d-987e-4ddb-aeb6-bf04c55cfe5c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.987756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81a26d-987e-4ddb-aeb6-bf04c55cfe5c None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:37.989301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 810/1607] 10.222.0.91 () {54 vars in 912 bytes} [Thu Sep 8 13:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:59:37.990008 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e76eac-d070-4183-b3b2-1bc44bde2241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:37.990151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81a26d-987e-4ddb-aeb6-bf04c55cfe5c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:37.990778 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e76eac-d070-4183-b3b2-1bc44bde2241 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:37.991250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e76eac-d070-4183-b3b2-1bc44bde2241 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:37.991549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e76eac-d070-4183-b3b2-1bc44bde2241 admin admin] PATH_INFO: `/v3/projects/b5693cbae8eb4f34a7a7b216223a3d1f/users/633dc1261eee47e19848aa4a106b2f59/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.005145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81a26d-987e-4ddb-aeb6-bf04c55cfe5c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:38.009751 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e76eac-d070-4183-b3b2-1bc44bde2241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5693cbae8eb4f34a7a7b216223a3d1f, user_id=633dc1261eee47e19848aa4a106b2f59, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:38.009990 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8809b6-be16-431e-b5e5-fbbee0eb5027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:38.010670 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8809b6-be16-431e-b5e5-fbbee0eb5027 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.010980 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8809b6-be16-431e-b5e5-fbbee0eb5027 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.011784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8809b6-be16-431e-b5e5-fbbee0eb5027 admin admin] PATH_INFO: `/v3/projects/681592d8b63a438c8b4893eca9a3f04b/users/beb41eb66e8140a6999b0bddab3c4afd/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.013199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e76eac-d070-4183-b3b2-1bc44bde2241 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:38.029699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8809b6-be16-431e-b5e5-fbbee0eb5027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=681592d8b63a438c8b4893eca9a3f04b, user_id=beb41eb66e8140a6999b0bddab3c4afd, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:38.032422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8809b6-be16-431e-b5e5-fbbee0eb5027 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:38.078742 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 798/1608] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:37 2022] PUT /identity/v3/projects/b5693cbae8eb4f34a7a7b216223a3d1f/users/633dc1261eee47e19848aa4a106b2f59/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:38.079698 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e76eac-d070-4183-b3b2-1bc44bde2241 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:38.101909 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 811/1609] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:37 2022] PUT /identity/v3/projects/681592d8b63a438c8b4893eca9a3f04b/users/beb41eb66e8140a6999b0bddab3c4afd/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:38.103508 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfad9f-87b3-4388-92d7-d044ba8986a6 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.103895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfad9f-87b3-4388-92d7-d044ba8986a6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.104319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfad9f-87b3-4388-92d7-d044ba8986a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.125822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d1dfad9f-87b3-4388-92d7-d044ba8986a6 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:38.127245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d1dfad9f-87b3-4388-92d7-d044ba8986a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:38.139331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e76eac-d070-4183-b3b2-1bc44bde2241 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:38.161977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:38.201161 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 812/1610] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:38.201437 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427884b-ff63-4b43-a8ab-8c797e165692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:38.201437 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427884b-ff63-4b43-a8ab-8c797e165692 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.201437 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427884b-ff63-4b43-a8ab-8c797e165692 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.201437 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427884b-ff63-4b43-a8ab-8c797e165692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.202129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd2bfd-b8a4-45ab-a118-6b8048752a6b None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.202129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd2bfd-b8a4-45ab-a118-6b8048752a6b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.202129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd2bfd-b8a4-45ab-a118-6b8048752a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.211437 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427884b-ff63-4b43-a8ab-8c797e165692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:38.212823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427884b-ff63-4b43-a8ab-8c797e165692 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:38.223335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-95cd2bfd-b8a4-45ab-a118-6b8048752a6b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:38.224539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-95cd2bfd-b8a4-45ab-a118-6b8048752a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:38.224795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 799/1611] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:38.226165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427884b-ff63-4b43-a8ab-8c797e165692 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:38.235451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427884b-ff63-4b43-a8ab-8c797e165692 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:38.237093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 813/1612] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:38.238001 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd2bfd-b8a4-45ab-a118-6b8048752a6b None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:38.249329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd2bfd-b8a4-45ab-a118-6b8048752a6b None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:38.252508 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91f33b-36e6-4839-8590-8fb37c6439ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:38.253183 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91f33b-36e6-4839-8590-8fb37c6439ef admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.253464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91f33b-36e6-4839-8590-8fb37c6439ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.253773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91f33b-36e6-4839-8590-8fb37c6439ef admin admin] PATH_INFO: `/v3/projects/5ba63eb7ead744f283bed215c7d25bda/users/8161f28a4a1845c280cbe1c9f4cd674a/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.254519 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef45fbf-cce4-4db1-be97-e49cf721b72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:38.255286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef45fbf-cce4-4db1-be97-e49cf721b72e admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.255599 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef45fbf-cce4-4db1-be97-e49cf721b72e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.255880 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef45fbf-cce4-4db1-be97-e49cf721b72e admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.256515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef45fbf-cce4-4db1-be97-e49cf721b72e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:38.258459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef45fbf-cce4-4db1-be97-e49cf721b72e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:38.265563 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:38.271446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91f33b-36e6-4839-8590-8fb37c6439ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba63eb7ead744f283bed215c7d25bda, user_id=8161f28a4a1845c280cbe1c9f4cd674a, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:38.275608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91f33b-36e6-4839-8590-8fb37c6439ef admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:38.399584 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 814/1613] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:38 2022] POST /identity/v3/projects => generated 403 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:38.403257 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a9b3b-2975-41a7-954f-41dec0b364d8 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.403257 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a9b3b-2975-41a7-954f-41dec0b364d8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.403257 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a9b3b-2975-41a7-954f-41dec0b364d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.426951 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 800/1614] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:38 2022] PUT /identity/v3/projects/5ba63eb7ead744f283bed215c7d25bda/users/8161f28a4a1845c280cbe1c9f4cd674a/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:38.428273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f64741-85a3-42a9-839e-ca3b31a2bed6 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.430211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-642a9b3b-2975-41a7-954f-41dec0b364d8 None None] MFA Rules not processed for user `38127f41852d412ea6ad270f1618fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:38.430453 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f64741-85a3-42a9-839e-ca3b31a2bed6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.430453 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f64741-85a3-42a9-839e-ca3b31a2bed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.430844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-642a9b3b-2975-41a7-954f-41dec0b364d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:38.462173 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-39f64741-85a3-42a9-839e-ca3b31a2bed6 None None] MFA Rules not processed for user `633dc1261eee47e19848aa4a106b2f59`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:38.463729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-39f64741-85a3-42a9-839e-ca3b31a2bed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:38.515304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 815/1615] 10.222.0.91 () {58 vars in 1009 bytes} [Thu Sep 8 13:59:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:38.517077 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba0763-6357-40e7-84dc-e7018b36329d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.518103 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba0763-6357-40e7-84dc-e7018b36329d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.518388 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba0763-6357-40e7-84dc-e7018b36329d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.559375 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-99ba0763-6357-40e7-84dc-e7018b36329d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:38.559375 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-99ba0763-6357-40e7-84dc-e7018b36329d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:38.559979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 801/1616] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:38.559979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f64741-85a3-42a9-839e-ca3b31a2bed6 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:38.628212 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 816/1617] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:38.628592 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f64741-85a3-42a9-839e-ca3b31a2bed6 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:38.630510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba0763-6357-40e7-84dc-e7018b36329d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:38.639888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ae91f-07b9-446a-8118-a8f5c523fb31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:38.641067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ae91f-07b9-446a-8118-a8f5c523fb31 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.641067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ae91f-07b9-446a-8118-a8f5c523fb31 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.641067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ae91f-07b9-446a-8118-a8f5c523fb31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.642111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba0763-6357-40e7-84dc-e7018b36329d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:38.645770 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22497416-48a1-444a-8d31-03b10778ec0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:38.646389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22497416-48a1-444a-8d31-03b10778ec0f service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.646695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22497416-48a1-444a-8d31-03b10778ec0f service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.647018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22497416-48a1-444a-8d31-03b10778ec0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.652922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ae91f-07b9-446a-8118-a8f5c523fb31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:38.654397 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ae91f-07b9-446a-8118-a8f5c523fb31 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:38.658477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22497416-48a1-444a-8d31-03b10778ec0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:38.660091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22497416-48a1-444a-8d31-03b10778ec0f service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:38.666243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 802/1618] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:38.667285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ae91f-07b9-446a-8118-a8f5c523fb31 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:38.670744 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 817/1619] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:38.672079 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20253c8e-5c71-4ff2-ae91-7c7b80160f31 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.672405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20253c8e-5c71-4ff2-ae91-7c7b80160f31 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.672751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20253c8e-5c71-4ff2-ae91-7c7b80160f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.679007 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ae91f-07b9-446a-8118-a8f5c523fb31 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:38.686025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c8ecd-3f6f-4a9b-a046-9584a0e129f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:38.687439 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c8ecd-3f6f-4a9b-a046-9584a0e129f4 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.687890 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c8ecd-3f6f-4a9b-a046-9584a0e129f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.688335 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c8ecd-3f6f-4a9b-a046-9584a0e129f4 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.689131 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c8ecd-3f6f-4a9b-a046-9584a0e129f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:38.691272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0c8ecd-3f6f-4a9b-a046-9584a0e129f4 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:38.693766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-20253c8e-5c71-4ff2-ae91-7c7b80160f31 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:38.695016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-20253c8e-5c71-4ff2-ae91-7c7b80160f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:38.704621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 818/1620] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:38.705389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20253c8e-5c71-4ff2-ae91-7c7b80160f31 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:38.776831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20253c8e-5c71-4ff2-ae91-7c7b80160f31 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:38.781342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06550abb-72df-4260-a60b-7753417d58e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:38.781969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06550abb-72df-4260-a60b-7753417d58e3 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.782369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06550abb-72df-4260-a60b-7753417d58e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.782507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06550abb-72df-4260-a60b-7753417d58e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.795698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06550abb-72df-4260-a60b-7753417d58e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:38.797595 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06550abb-72df-4260-a60b-7753417d58e3 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:38.809706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 819/1621] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:38.823059 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06550abb-72df-4260-a60b-7753417d58e3 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:38.823493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 803/1622] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:38 2022] POST /identity/v3/users => generated 308 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:38.825000 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c8ecd-3f6f-4a9b-a046-9584a0e129f4 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:38.832341 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:38.839452 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06550abb-72df-4260-a60b-7753417d58e3 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:38.839978 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c8ecd-3f6f-4a9b-a046-9584a0e129f4 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:38.849120 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b30db-7080-4000-a166-8345ada36186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:38.849752 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b30db-7080-4000-a166-8345ada36186 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.850037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b30db-7080-4000-a166-8345ada36186 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.850372 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b30db-7080-4000-a166-8345ada36186 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.851055 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b30db-7080-4000-a166-8345ada36186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:38.853532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b30db-7080-4000-a166-8345ada36186 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:38.868414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d9150-a584-481a-92bd-75528c0a4014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:38.869325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d9150-a584-481a-92bd-75528c0a4014 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.869810 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d9150-a584-481a-92bd-75528c0a4014 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.870288 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d9150-a584-481a-92bd-75528c0a4014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.898399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d9150-a584-481a-92bd-75528c0a4014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:38.900315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d9150-a584-481a-92bd-75528c0a4014 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:38.918237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:38.929794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 804/1623] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:38.937909 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d9150-a584-481a-92bd-75528c0a4014 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:38.965384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d9150-a584-481a-92bd-75528c0a4014 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:38.972818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a76d4e-7f3a-4237-818d-78c4735ddb50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:38.973775 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a76d4e-7f3a-4237-818d-78c4735ddb50 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.974660 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a76d4e-7f3a-4237-818d-78c4735ddb50 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.975221 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a76d4e-7f3a-4237-818d-78c4735ddb50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.989638 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a76d4e-7f3a-4237-818d-78c4735ddb50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:38.992859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a76d4e-7f3a-4237-818d-78c4735ddb50 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:38.993014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 820/1624] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:38 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:38.993424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b30db-7080-4000-a166-8345ada36186 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:39.003250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 805/1625] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:39.004118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a76d4e-7f3a-4237-818d-78c4735ddb50 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:39.015633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a76d4e-7f3a-4237-818d-78c4735ddb50 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:39.082664 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b30db-7080-4000-a166-8345ada36186 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:39.091412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7deaee-2335-45b2-83df-252ec97dcc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:39.092091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7deaee-2335-45b2-83df-252ec97dcc3b admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:39.092357 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f9321-57ac-4277-85fd-39b178bc0bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:39.092499 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7deaee-2335-45b2-83df-252ec97dcc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:39.092632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7deaee-2335-45b2-83df-252ec97dcc3b admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:39.093579 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f9321-57ac-4277-85fd-39b178bc0bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:39.093809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7deaee-2335-45b2-83df-252ec97dcc3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:39.094104 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f9321-57ac-4277-85fd-39b178bc0bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:39.094457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f9321-57ac-4277-85fd-39b178bc0bb7 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:39.095633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f9321-57ac-4277-85fd-39b178bc0bb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:39.101087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7deaee-2335-45b2-83df-252ec97dcc3b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:39.101501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f9321-57ac-4277-85fd-39b178bc0bb7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:39.110253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 821/1626] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:39.115205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7deaee-2335-45b2-83df-252ec97dcc3b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:39.132710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7deaee-2335-45b2-83df-252ec97dcc3b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:39.144705 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364e247-6a01-4dbf-a55b-8fd64b4a2eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:39.145959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364e247-6a01-4dbf-a55b-8fd64b4a2eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:39.146521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364e247-6a01-4dbf-a55b-8fd64b4a2eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:39.147073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364e247-6a01-4dbf-a55b-8fd64b4a2eb5 admin admin] PATH_INFO: `/v3/projects/e89d7c46dbfc4b69b634d797938703db/users/ebdd587721474313a9ad6a036e9e79dc/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:39.193606 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 806/1627] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:39 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:39.194100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f9321-57ac-4277-85fd-39b178bc0bb7 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:39.198801 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364e247-6a01-4dbf-a55b-8fd64b4a2eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e89d7c46dbfc4b69b634d797938703db, user_id=ebdd587721474313a9ad6a036e9e79dc, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:39.203840 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364e247-6a01-4dbf-a55b-8fd64b4a2eb5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:39.214466 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f9321-57ac-4277-85fd-39b178bc0bb7 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:39.228298 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6aba9e-2e17-4071-9b89-d98b7b068b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:39.230730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6aba9e-2e17-4071-9b89-d98b7b068b4a admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:39.231262 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6aba9e-2e17-4071-9b89-d98b7b068b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:39.231815 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6aba9e-2e17-4071-9b89-d98b7b068b4a admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:39.232955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6aba9e-2e17-4071-9b89-d98b7b068b4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:39.235232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6aba9e-2e17-4071-9b89-d98b7b068b4a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:39.251828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 807/1628] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:59:39.257671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6aba9e-2e17-4071-9b89-d98b7b068b4a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:39.273482 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6aba9e-2e17-4071-9b89-d98b7b068b4a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:39.279236 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ccbae-10f4-4a7c-ae2c-ab3910a8c7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:39.279992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ccbae-10f4-4a7c-ae2c-ab3910a8c7f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:39.280265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ccbae-10f4-4a7c-ae2c-ab3910a8c7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:39.280533 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ccbae-10f4-4a7c-ae2c-ab3910a8c7f8 admin admin] PATH_INFO: `/v3/projects/624feb8c32d24b0781cc585c797c01e5/users/5ad3d83a74ef4472acb2422658775c66/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:39.302737 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ccbae-10f4-4a7c-ae2c-ab3910a8c7f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=624feb8c32d24b0781cc585c797c01e5, user_id=5ad3d83a74ef4472acb2422658775c66, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:39.306657 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ccbae-10f4-4a7c-ae2c-ab3910a8c7f8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:39.332225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 822/1629] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:39 2022] PUT /identity/v3/projects/e89d7c46dbfc4b69b634d797938703db/users/ebdd587721474313a9ad6a036e9e79dc/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:59:39.336108 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364e247-6a01-4dbf-a55b-8fd64b4a2eb5 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:39.424223 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364e247-6a01-4dbf-a55b-8fd64b4a2eb5 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:39.430508 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fdbcb-d1dd-42bf-9805-a33e1c2cda84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:39.431841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fdbcb-d1dd-42bf-9805-a33e1c2cda84 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:39.432477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fdbcb-d1dd-42bf-9805-a33e1c2cda84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:39.433737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fdbcb-d1dd-42bf-9805-a33e1c2cda84 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:39.434932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fdbcb-d1dd-42bf-9805-a33e1c2cda84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:39.436950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fdbcb-d1dd-42bf-9805-a33e1c2cda84 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:39.457602 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 808/1631] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:39 2022] PUT /identity/v3/projects/624feb8c32d24b0781cc585c797c01e5/users/5ad3d83a74ef4472acb2422658775c66/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:59:39.462705 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 823/1631] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:59:39.462849 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ccbae-10f4-4a7c-ae2c-ab3910a8c7f8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:39.462961 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fdbcb-d1dd-42bf-9805-a33e1c2cda84 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:39.475561 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fdbcb-d1dd-42bf-9805-a33e1c2cda84 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:39.546603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:39.561895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920818f4-c1a8-4bfb-938d-145e30c18b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:39.562431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920818f4-c1a8-4bfb-938d-145e30c18b47 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:39.562731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920818f4-c1a8-4bfb-938d-145e30c18b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:39.563070 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920818f4-c1a8-4bfb-938d-145e30c18b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:39.563416 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ccbae-10f4-4a7c-ae2c-ab3910a8c7f8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:39.563750 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920818f4-c1a8-4bfb-938d-145e30c18b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:39.565422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920818f4-c1a8-4bfb-938d-145e30c18b47 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:39.571354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e268e-1bb6-4c21-bc36-ce48b465dce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:39.572142 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e268e-1bb6-4c21-bc36-ce48b465dce5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:39.572492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e268e-1bb6-4c21-bc36-ce48b465dce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:39.572960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e268e-1bb6-4c21-bc36-ce48b465dce5 admin admin] PATH_INFO: `/v3/projects/e89d7c46dbfc4b69b634d797938703db/users/ebdd587721474313a9ad6a036e9e79dc/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:39.575518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e268e-1bb6-4c21-bc36-ce48b465dce5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e89d7c46dbfc4b69b634d797938703db, user_id=ebdd587721474313a9ad6a036e9e79dc, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:39.578802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 824/1632] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:39.580338 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e268e-1bb6-4c21-bc36-ce48b465dce5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:39.746388 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920818f4-c1a8-4bfb-938d-145e30c18b47 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:39.747486 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 809/1633] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:39 2022] PUT /identity/v3/projects/e89d7c46dbfc4b69b634d797938703db/users/ebdd587721474313a9ad6a036e9e79dc/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:39.749395 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920818f4-c1a8-4bfb-938d-145e30c18b47 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:39.749902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b5f67-423d-42e3-8c23-0905d5bc5503 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:39.749902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b5f67-423d-42e3-8c23-0905d5bc5503 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:39.749902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b5f67-423d-42e3-8c23-0905d5bc5503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:39.749902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-811b5f67-423d-42e3-8c23-0905d5bc5503 None None] MFA Rules not processed for user `ebdd587721474313a9ad6a036e9e79dc`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:39.749902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-811b5f67-423d-42e3-8c23-0905d5bc5503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:39.757453 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2410551-e6d1-4083-967c-123614d272b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:39.759543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2410551-e6d1-4083-967c-123614d272b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:39.760410 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2410551-e6d1-4083-967c-123614d272b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:39.761378 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2410551-e6d1-4083-967c-123614d272b5 admin admin] PATH_INFO: `/v3/projects/624feb8c32d24b0781cc585c797c01e5/users/5ad3d83a74ef4472acb2422658775c66/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:39.764178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:39.765072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2410551-e6d1-4083-967c-123614d272b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=624feb8c32d24b0781cc585c797c01e5, user_id=5ad3d83a74ef4472acb2422658775c66, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:39.769261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2410551-e6d1-4083-967c-123614d272b5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:39.796641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 825/1634] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:39 2022] PUT /identity/v3/projects/624feb8c32d24b0781cc585c797c01e5/users/5ad3d83a74ef4472acb2422658775c66/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:39.814041 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d65b8-223a-4343-8545-fa9f7f150e0a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:39.814421 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d65b8-223a-4343-8545-fa9f7f150e0a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:39.814711 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d65b8-223a-4343-8545-fa9f7f150e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:39.867086 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-617d65b8-223a-4343-8545-fa9f7f150e0a None None] MFA Rules not processed for user `5ad3d83a74ef4472acb2422658775c66`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:39.868988 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-617d65b8-223a-4343-8545-fa9f7f150e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:39.914910 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 810/1635] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:39 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:39.920224 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b5f67-423d-42e3-8c23-0905d5bc5503 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:39.981573 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 826/1636] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:59:40.000996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b5f67-423d-42e3-8c23-0905d5bc5503 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:40.112894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb846381-52e9-41f1-9d93-9cf7ebaf0683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:40.113671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb846381-52e9-41f1-9d93-9cf7ebaf0683 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:40.114184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb846381-52e9-41f1-9d93-9cf7ebaf0683 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:40.114346 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb846381-52e9-41f1-9d93-9cf7ebaf0683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:40.202824 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb846381-52e9-41f1-9d93-9cf7ebaf0683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:40.204461 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb846381-52e9-41f1-9d93-9cf7ebaf0683 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:40.217454 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 811/1637] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:39 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:59:40.276243 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d65b8-223a-4343-8545-fa9f7f150e0a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:40.288274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d65b8-223a-4343-8545-fa9f7f150e0a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:40.294207 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a85d28-1c54-4214-92bb-d4e54e8da987 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:40.295124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a85d28-1c54-4214-92bb-d4e54e8da987 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:40.295513 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a85d28-1c54-4214-92bb-d4e54e8da987 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:40.296036 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a85d28-1c54-4214-92bb-d4e54e8da987 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:40.318753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a85d28-1c54-4214-92bb-d4e54e8da987 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:40.321351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a85d28-1c54-4214-92bb-d4e54e8da987 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:40.344704 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 827/1638] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:40.723027 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb846381-52e9-41f1-9d93-9cf7ebaf0683 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:40.805981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:40.889897 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb846381-52e9-41f1-9d93-9cf7ebaf0683 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:40.897137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa397c-bb29-489f-acdd-8ca451896b6c None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:40.898024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa397c-bb29-489f-acdd-8ca451896b6c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:40.898399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26fbf4-492a-4ef1-9192-a6de4037cb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:40.898626 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa397c-bb29-489f-acdd-8ca451896b6c None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:40.901167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 828/1639] 10.222.0.91 () {54 vars in 962 bytes} [Thu Sep 8 13:59:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:59:40.909891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26fbf4-492a-4ef1-9192-a6de4037cb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:40.910397 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26fbf4-492a-4ef1-9192-a6de4037cb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:40.911217 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26fbf4-492a-4ef1-9192-a6de4037cb58 admin admin] PATH_INFO: `/v3/users/30b267a6fd414391a33389a481de607d` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:40.913016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26fbf4-492a-4ef1-9192-a6de4037cb58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30b267a6fd414391a33389a481de607d)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:40.914379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: WARNING py.warnings [None req-3f26fbf4-492a-4ef1-9192-a6de4037cb58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:59:40.914379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: warnings.warn(msg) Sep 08 13:59:40.914379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: Sep 08 13:59:40.924866 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45982605-a8b1-44d1-9761-ab865bf1a74d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:40.925510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45982605-a8b1-44d1-9761-ab865bf1a74d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:40.925949 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26fbf4-492a-4ef1-9192-a6de4037cb58 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:40.926447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45982605-a8b1-44d1-9761-ab865bf1a74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:40.972244 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-45982605-a8b1-44d1-9761-ab865bf1a74d None None] MFA Rules not processed for user `38127f41852d412ea6ad270f1618fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:40.974411 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-45982605-a8b1-44d1-9761-ab865bf1a74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:41.072128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:41.111301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 829/1640] 10.222.0.91 () {58 vars in 1059 bytes} [Thu Sep 8 13:59:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:59:41.120505 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1231e7-0c4d-40f4-815a-2ef3a0c32fa0 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:41.121137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1231e7-0c4d-40f4-815a-2ef3a0c32fa0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:41.121809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1231e7-0c4d-40f4-815a-2ef3a0c32fa0 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:41.123638 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 830/1641] 10.222.0.91 () {54 vars in 912 bytes} [Thu Sep 8 13:59:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:59:41.132517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6855fa5-c9da-42ec-ba1e-8f6ce0ebbf82 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:41.132517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6855fa5-c9da-42ec-ba1e-8f6ce0ebbf82 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:41.132517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6855fa5-c9da-42ec-ba1e-8f6ce0ebbf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:41.163726 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c6855fa5-c9da-42ec-ba1e-8f6ce0ebbf82 None None] MFA Rules not processed for user `38127f41852d412ea6ad270f1618fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:41.163726 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c6855fa5-c9da-42ec-ba1e-8f6ce0ebbf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:41.179776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 831/1642] 10.222.0.91 () {58 vars in 1009 bytes} [Thu Sep 8 13:59:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:41.190879 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6855fa5-c9da-42ec-ba1e-8f6ce0ebbf82 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:41.216703 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6855fa5-c9da-42ec-ba1e-8f6ce0ebbf82 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:41.223385 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32935fbb-1218-4c80-9e50-82f3a1f22a31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:41.224043 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32935fbb-1218-4c80-9e50-82f3a1f22a31 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:41.224365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32935fbb-1218-4c80-9e50-82f3a1f22a31 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:41.224606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32935fbb-1218-4c80-9e50-82f3a1f22a31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:41.227472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3f26fbf4-492a-4ef1-9192-a6de4037cb58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30b267a6fd414391a33389a481de607d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:41.238019 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32935fbb-1218-4c80-9e50-82f3a1f22a31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:41.239535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32935fbb-1218-4c80-9e50-82f3a1f22a31 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:41.251975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 832/1643] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:59:41.341504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3f26fbf4-492a-4ef1-9192-a6de4037cb58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30b267a6fd414391a33389a481de607d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:41.369152 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3f26fbf4-492a-4ef1-9192-a6de4037cb58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30b267a6fd414391a33389a481de607d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:41.370209 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3f26fbf4-492a-4ef1-9192-a6de4037cb58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30b267a6fd414391a33389a481de607d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:41.386835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3f26fbf4-492a-4ef1-9192-a6de4037cb58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30b267a6fd414391a33389a481de607d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:41.398564 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 812/1644] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 13:59:40 2022] DELETE /identity/v3/users/30b267a6fd414391a33389a481de607d => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:41.403995 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9fb5f-4882-47bf-944f-2b1e449afa7f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:41.404510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9fb5f-4882-47bf-944f-2b1e449afa7f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:41.404958 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9fb5f-4882-47bf-944f-2b1e449afa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:41.439683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-aab9fb5f-4882-47bf-944f-2b1e449afa7f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:41.441894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-aab9fb5f-4882-47bf-944f-2b1e449afa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:41.534376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 833/1645] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:41.545150 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26fbf4-492a-4ef1-9192-a6de4037cb58 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:41.546942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3f26fbf4-492a-4ef1-9192-a6de4037cb58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:41.665609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26fbf4-492a-4ef1-9192-a6de4037cb58 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:41.670310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564db14-4577-4485-859d-c25303babbaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:41.671733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564db14-4577-4485-859d-c25303babbaf service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:41.672121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564db14-4577-4485-859d-c25303babbaf service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:41.672586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564db14-4577-4485-859d-c25303babbaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:41.687374 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564db14-4577-4485-859d-c25303babbaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:41.689520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564db14-4577-4485-859d-c25303babbaf service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:41.718626 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 813/1646] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:41.929957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9fb5f-4882-47bf-944f-2b1e449afa7f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:41.935452 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-aab9fb5f-4882-47bf-944f-2b1e449afa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:41.967563 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9fb5f-4882-47bf-944f-2b1e449afa7f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:41.974554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ed64a-50af-4f4f-8c93-bff3fdbf7705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:41.975552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ed64a-50af-4f4f-8c93-bff3fdbf7705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:41.976053 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ed64a-50af-4f4f-8c93-bff3fdbf7705 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:41.976543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ed64a-50af-4f4f-8c93-bff3fdbf7705 admin admin] PATH_INFO: `/v3/users/633dc1261eee47e19848aa4a106b2f59` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:41.978672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ed64a-50af-4f4f-8c93-bff3fdbf7705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=633dc1261eee47e19848aa4a106b2f59)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:41.981099 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ed64a-50af-4f4f-8c93-bff3fdbf7705 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:42.163527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d8268-53b9-4d3d-bab9-dff9c6441350 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:42.163984 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d8268-53b9-4d3d-bab9-dff9c6441350 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:42.164068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d8268-53b9-4d3d-bab9-dff9c6441350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:42.210645 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-885d8268-53b9-4d3d-bab9-dff9c6441350 None None] MFA Rules not processed for user `8161f28a4a1845c280cbe1c9f4cd674a`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:42.212698 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-885d8268-53b9-4d3d-bab9-dff9c6441350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:42.221500 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-044ed64a-50af-4f4f-8c93-bff3fdbf7705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '633dc1261eee47e19848aa4a106b2f59'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:42.275599 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:42.289119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-044ed64a-50af-4f4f-8c93-bff3fdbf7705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '633dc1261eee47e19848aa4a106b2f59'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:42.290373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-044ed64a-50af-4f4f-8c93-bff3fdbf7705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '633dc1261eee47e19848aa4a106b2f59'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:42.303278 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-044ed64a-50af-4f4f-8c93-bff3fdbf7705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '633dc1261eee47e19848aa4a106b2f59'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:42.330151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-044ed64a-50af-4f4f-8c93-bff3fdbf7705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '633dc1261eee47e19848aa4a106b2f59'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:42.347800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 834/1647] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 13:59:41 2022] DELETE /identity/v3/users/633dc1261eee47e19848aa4a106b2f59 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:42.349176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3388a71-889e-4bce-8e3f-7957d3540a24 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:42.349465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3388a71-889e-4bce-8e3f-7957d3540a24 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:42.349742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3388a71-889e-4bce-8e3f-7957d3540a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:42.361630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 814/1648] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:42 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:59:42.366242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d8268-53b9-4d3d-bab9-dff9c6441350 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:42.369478 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-885d8268-53b9-4d3d-bab9-dff9c6441350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:42.392158 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f3388a71-889e-4bce-8e3f-7957d3540a24 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:42.392158 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f3388a71-889e-4bce-8e3f-7957d3540a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:42.428449 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d8268-53b9-4d3d-bab9-dff9c6441350 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:42.548203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 835/1649] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:42.592878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3388a71-889e-4bce-8e3f-7957d3540a24 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:42.594454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f3388a71-889e-4bce-8e3f-7957d3540a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:42.596544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7f0e4-a60f-4407-ab76-5d517cffbc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:42.597202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7f0e4-a60f-4407-ab76-5d517cffbc9a admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:42.597480 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7f0e4-a60f-4407-ab76-5d517cffbc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:42.597756 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7f0e4-a60f-4407-ab76-5d517cffbc9a admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:42.598733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7f0e4-a60f-4407-ab76-5d517cffbc9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:42.612792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7f0e4-a60f-4407-ab76-5d517cffbc9a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:42.723436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3388a71-889e-4bce-8e3f-7957d3540a24 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:42.727713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da380a-eaa9-4971-af8e-66fabc7423ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:42.728593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da380a-eaa9-4971-af8e-66fabc7423ac service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:42.728872 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da380a-eaa9-4971-af8e-66fabc7423ac service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:42.729143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da380a-eaa9-4971-af8e-66fabc7423ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:42.749648 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 815/1650] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:42 2022] POST /identity/v3/projects => generated 405 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:59:42.751540 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7f0e4-a60f-4407-ab76-5d517cffbc9a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:42.767299 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da380a-eaa9-4971-af8e-66fabc7423ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:42.771428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da380a-eaa9-4971-af8e-66fabc7423ac service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:42.816387 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:42.827597 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 836/1651] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:59:42.888446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da380a-eaa9-4971-af8e-66fabc7423ac service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:42.889000 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7f0e4-a60f-4407-ab76-5d517cffbc9a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:42.890967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-45da380a-eaa9-4971-af8e-66fabc7423ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:42.894574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e918c9-2fdb-4ecb-a8be-a229b7ca3c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:42.895482 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e918c9-2fdb-4ecb-a8be-a229b7ca3c9c admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:42.895716 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e918c9-2fdb-4ecb-a8be-a229b7ca3c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:42.896074 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e918c9-2fdb-4ecb-a8be-a229b7ca3c9c admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:42.896881 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e918c9-2fdb-4ecb-a8be-a229b7ca3c9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:42.899990 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e918c9-2fdb-4ecb-a8be-a229b7ca3c9c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:42.921040 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da380a-eaa9-4971-af8e-66fabc7423ac service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:42.928204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafb8a5-68f6-4afc-a900-fd421f31e762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:42.928917 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafb8a5-68f6-4afc-a900-fd421f31e762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:42.929484 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafb8a5-68f6-4afc-a900-fd421f31e762 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:42.929877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafb8a5-68f6-4afc-a900-fd421f31e762 admin admin] PATH_INFO: `/v3/projects/8ac10c5287324bfb80ed74e15efc5308` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:42.931233 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafb8a5-68f6-4afc-a900-fd421f31e762 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac10c5287324bfb80ed74e15efc5308)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:42.932413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: WARNING py.warnings [None req-7eafb8a5-68f6-4afc-a900-fd421f31e762 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:59:42.932413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: warnings.warn(msg) Sep 08 13:59:42.932413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: Sep 08 13:59:42.934789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafb8a5-68f6-4afc-a900-fd421f31e762 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:43.045053 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 816/1652] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:42 2022] POST /identity/v3/users => generated 309 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:59:43.046095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e918c9-2fdb-4ecb-a8be-a229b7ca3c9c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:43.072812 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e918c9-2fdb-4ecb-a8be-a229b7ca3c9c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:43.100614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a9b26-9028-4841-a5ce-3a148d75fb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:43.106302 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a9b26-9028-4841-a5ce-3a148d75fb11 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:43.107016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a9b26-9028-4841-a5ce-3a148d75fb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:43.107490 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a9b26-9028-4841-a5ce-3a148d75fb11 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:43.108378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a9b26-9028-4841-a5ce-3a148d75fb11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:43.112553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a9b26-9028-4841-a5ce-3a148d75fb11 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:43.145350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 817/1653] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:43.149281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a9b26-9028-4841-a5ce-3a148d75fb11 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:43.151170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7eafb8a5-68f6-4afc-a900-fd421f31e762 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac10c5287324bfb80ed74e15efc5308', 'request_id': 'req-7eafb8a5-68f6-4afc-a900-fd421f31e762'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:43.179691 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a9b26-9028-4841-a5ce-3a148d75fb11 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:43.237682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7eafb8a5-68f6-4afc-a900-fd421f31e762 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac10c5287324bfb80ed74e15efc5308', 'request_id': 'req-7eafb8a5-68f6-4afc-a900-fd421f31e762'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:43.265300 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7eafb8a5-68f6-4afc-a900-fd421f31e762 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac10c5287324bfb80ed74e15efc5308', 'request_id': 'req-7eafb8a5-68f6-4afc-a900-fd421f31e762'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:43.290818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275880c2-a954-458d-8c24-32098a405b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:43.291042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7eafb8a5-68f6-4afc-a900-fd421f31e762 admin admin] The token cache is being invalidate because project 8ac10c5287324bfb80ed74e15efc5308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:59:43.292063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275880c2-a954-458d-8c24-32098a405b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:43.293265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275880c2-a954-458d-8c24-32098a405b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:43.293566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7eafb8a5-68f6-4afc-a900-fd421f31e762 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:43.294834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7eafb8a5-68f6-4afc-a900-fd421f31e762 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:43.297519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275880c2-a954-458d-8c24-32098a405b9c admin admin] PATH_INFO: `/v3/projects/516db22260e64e888a2c06164c83e3f0/users/a8aafe3997c144c08edd2b829ae51135/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:43.298284 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 837/1654] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 13:59:42 2022] DELETE /identity/v3/projects/8ac10c5287324bfb80ed74e15efc5308 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:59:43.302233 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafb8a5-68f6-4afc-a900-fd421f31e762 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:43.313206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7eafb8a5-68f6-4afc-a900-fd421f31e762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:43.328471 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275880c2-a954-458d-8c24-32098a405b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=516db22260e64e888a2c06164c83e3f0, user_id=a8aafe3997c144c08edd2b829ae51135, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:43.331933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275880c2-a954-458d-8c24-32098a405b9c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:43.349563 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafb8a5-68f6-4afc-a900-fd421f31e762 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:43.362091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b534be76-fcd6-4961-8164-3a7abe4718f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:43.362969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534be76-fcd6-4961-8164-3a7abe4718f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:43.363342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534be76-fcd6-4961-8164-3a7abe4718f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:43.363708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b534be76-fcd6-4961-8164-3a7abe4718f6 admin admin] PATH_INFO: `/v3/users/ebdd587721474313a9ad6a036e9e79dc` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:43.367184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534be76-fcd6-4961-8164-3a7abe4718f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebdd587721474313a9ad6a036e9e79dc)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:43.373891 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b534be76-fcd6-4961-8164-3a7abe4718f6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:43.490719 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 818/1655] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:43 2022] PUT /identity/v3/projects/516db22260e64e888a2c06164c83e3f0/users/a8aafe3997c144c08edd2b829ae51135/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:59:43.494735 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275880c2-a954-458d-8c24-32098a405b9c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:43.498575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-275880c2-a954-458d-8c24-32098a405b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:43.691953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b534be76-fcd6-4961-8164-3a7abe4718f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebdd587721474313a9ad6a036e9e79dc'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:43.699205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275880c2-a954-458d-8c24-32098a405b9c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:43.705779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fe7d7-b691-4eb3-a18f-efd956f1f158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:43.706791 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fe7d7-b691-4eb3-a18f-efd956f1f158 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:43.706926 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fe7d7-b691-4eb3-a18f-efd956f1f158 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:43.707309 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fe7d7-b691-4eb3-a18f-efd956f1f158 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:43.708106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fe7d7-b691-4eb3-a18f-efd956f1f158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:43.710495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fe7d7-b691-4eb3-a18f-efd956f1f158 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:43.728985 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 819/1656] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:43.730713 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fe7d7-b691-4eb3-a18f-efd956f1f158 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:43.745955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fe7d7-b691-4eb3-a18f-efd956f1f158 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:43.754135 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84689b-2496-4f5d-948d-3e82ccbb8105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:43.755021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84689b-2496-4f5d-948d-3e82ccbb8105 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:43.755406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84689b-2496-4f5d-948d-3e82ccbb8105 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:43.755833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84689b-2496-4f5d-948d-3e82ccbb8105 admin admin] PATH_INFO: `/v3/projects/516db22260e64e888a2c06164c83e3f0/users/a8aafe3997c144c08edd2b829ae51135/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:43.759102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84689b-2496-4f5d-948d-3e82ccbb8105 admin admin] RBAC: Authorizing `identity:create_grant(project_id=516db22260e64e888a2c06164c83e3f0, user_id=a8aafe3997c144c08edd2b829ae51135, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:43.762770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84689b-2496-4f5d-948d-3e82ccbb8105 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:43.791278 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 820/1657] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:43 2022] PUT /identity/v3/projects/516db22260e64e888a2c06164c83e3f0/users/a8aafe3997c144c08edd2b829ae51135/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:43.814909 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b534be76-fcd6-4961-8164-3a7abe4718f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebdd587721474313a9ad6a036e9e79dc'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:43.816522 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b534be76-fcd6-4961-8164-3a7abe4718f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebdd587721474313a9ad6a036e9e79dc'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:43.828552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b534be76-fcd6-4961-8164-3a7abe4718f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebdd587721474313a9ad6a036e9e79dc'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:43.879623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b534be76-fcd6-4961-8164-3a7abe4718f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebdd587721474313a9ad6a036e9e79dc'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:43.896965 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 838/1658] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 13:59:43 2022] DELETE /identity/v3/users/ebdd587721474313a9ad6a036e9e79dc => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:44.177176 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84689b-2496-4f5d-948d-3e82ccbb8105 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:44.179553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-be84689b-2496-4f5d-948d-3e82ccbb8105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:44.375263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84689b-2496-4f5d-948d-3e82ccbb8105 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:44.559767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca2523-9f20-44a8-b69b-dfcb344a3c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:44.564751 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca2523-9f20-44a8-b69b-dfcb344a3c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:44.564966 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca2523-9f20-44a8-b69b-dfcb344a3c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:44.565249 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca2523-9f20-44a8-b69b-dfcb344a3c19 admin admin] PATH_INFO: `/v3/users/73601a32878240dd85194e733a671a86` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:44.572983 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca2523-9f20-44a8-b69b-dfcb344a3c19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73601a32878240dd85194e733a671a86)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:44.577321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca2523-9f20-44a8-b69b-dfcb344a3c19 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:44.722617 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:44.808693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-05ca2523-9f20-44a8-b69b-dfcb344a3c19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73601a32878240dd85194e733a671a86'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:44.896163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-05ca2523-9f20-44a8-b69b-dfcb344a3c19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73601a32878240dd85194e733a671a86'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:44.929266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-05ca2523-9f20-44a8-b69b-dfcb344a3c19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73601a32878240dd85194e733a671a86'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:44.930430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-05ca2523-9f20-44a8-b69b-dfcb344a3c19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73601a32878240dd85194e733a671a86'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:44.960836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-05ca2523-9f20-44a8-b69b-dfcb344a3c19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73601a32878240dd85194e733a671a86'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:44.983644 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 821/1659] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 13:59:44 2022] DELETE /identity/v3/users/73601a32878240dd85194e733a671a86 => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:44.984879 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645e53a-8c04-49fb-98bc-4080fcb2c764 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:44.985926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645e53a-8c04-49fb-98bc-4080fcb2c764 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:44.987058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645e53a-8c04-49fb-98bc-4080fcb2c764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:45.030667 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e645e53a-8c04-49fb-98bc-4080fcb2c764 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:45.032452 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e645e53a-8c04-49fb-98bc-4080fcb2c764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:45.197211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 839/1660] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:59:45.220257 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca2523-9f20-44a8-b69b-dfcb344a3c19 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:45.222520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-05ca2523-9f20-44a8-b69b-dfcb344a3c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:45.245446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645e53a-8c04-49fb-98bc-4080fcb2c764 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:45.247309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e645e53a-8c04-49fb-98bc-4080fcb2c764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:45.268630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645e53a-8c04-49fb-98bc-4080fcb2c764 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:45.289796 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:45.290748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:45.291152 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:45.291525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd admin admin] PATH_INFO: `/v3/projects/e89d7c46dbfc4b69b634d797938703db` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:45.293792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e89d7c46dbfc4b69b634d797938703db)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:45.297173 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:45.394841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:45.487296 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca2523-9f20-44a8-b69b-dfcb344a3c19 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:45.516057 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194b6e8-0ab2-4b66-a6dc-3805c88985d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:45.519562 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194b6e8-0ab2-4b66-a6dc-3805c88985d9 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:45.519979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194b6e8-0ab2-4b66-a6dc-3805c88985d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:45.520355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194b6e8-0ab2-4b66-a6dc-3805c88985d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:45.563065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194b6e8-0ab2-4b66-a6dc-3805c88985d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:45.566778 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194b6e8-0ab2-4b66-a6dc-3805c88985d9 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:45.593145 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 822/1661] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:45.597049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e89d7c46dbfc4b69b634d797938703db', 'request_id': 'req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:45.711924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e89d7c46dbfc4b69b634d797938703db', 'request_id': 'req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:45.730880 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e89d7c46dbfc4b69b634d797938703db', 'request_id': 'req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:45.767383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd admin admin] The token cache is being invalidate because project e89d7c46dbfc4b69b634d797938703db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:59:45.767383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:45.768191 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:45.770205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 840/1662] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 13:59:45 2022] DELETE /identity/v3/projects/e89d7c46dbfc4b69b634d797938703db => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:45.776863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3894feb-75c2-480e-9e3a-cd7b999e4efe None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:45.777298 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3894feb-75c2-480e-9e3a-cd7b999e4efe None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:45.778349 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3894feb-75c2-480e-9e3a-cd7b999e4efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:45.827557 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a3894feb-75c2-480e-9e3a-cd7b999e4efe None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:45.833996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a3894feb-75c2-480e-9e3a-cd7b999e4efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:45.874273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:45.875839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:46.062627 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 823/1663] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:46.064585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80a1ad-d982-44e3-8a12-d831a02f42b5 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:46.065006 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80a1ad-d982-44e3-8a12-d831a02f42b5 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:46.065384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80a1ad-d982-44e3-8a12-d831a02f42b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:46.112356 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9a80a1ad-d982-44e3-8a12-d831a02f42b5 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:46.115527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9a80a1ad-d982-44e3-8a12-d831a02f42b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:46.138723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760d4e6-3297-43b6-9ddd-bcbca114e4dd admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:46.140381 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 824/1664] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:46.141459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80a1ad-d982-44e3-8a12-d831a02f42b5 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:46.156116 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3d105-4625-4410-9005-bfc1eb1c9580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:46.157039 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3d105-4625-4410-9005-bfc1eb1c9580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:46.157579 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3d105-4625-4410-9005-bfc1eb1c9580 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:46.158302 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3d105-4625-4410-9005-bfc1eb1c9580 admin admin] PATH_INFO: `/v3/projects/b5693cbae8eb4f34a7a7b216223a3d1f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:46.159650 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a3d105-4625-4410-9005-bfc1eb1c9580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5693cbae8eb4f34a7a7b216223a3d1f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:46.161994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a3d105-4625-4410-9005-bfc1eb1c9580 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:46.163043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80a1ad-d982-44e3-8a12-d831a02f42b5 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:46.172580 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e51d2-43ad-4601-b212-02d1cc5aa9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:46.173697 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e51d2-43ad-4601-b212-02d1cc5aa9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:46.177550 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e51d2-43ad-4601-b212-02d1cc5aa9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:46.177976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e51d2-43ad-4601-b212-02d1cc5aa9d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:46.178769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e51d2-43ad-4601-b212-02d1cc5aa9d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:59:46.179199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e51d2-43ad-4601-b212-02d1cc5aa9d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:46.180932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292e51d2-43ad-4601-b212-02d1cc5aa9d9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:46.213649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 825/1665] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 13:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:59:46.219570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e51d2-43ad-4601-b212-02d1cc5aa9d9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:46.238167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e51d2-43ad-4601-b212-02d1cc5aa9d9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:46.250933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3403b-3c12-4b63-ba98-c261119e42ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:46.251825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3403b-3c12-4b63-ba98-c261119e42ab admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:46.252294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3403b-3c12-4b63-ba98-c261119e42ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:46.252779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3403b-3c12-4b63-ba98-c261119e42ab admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:46.253915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3403b-3c12-4b63-ba98-c261119e42ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:46.256201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3403b-3c12-4b63-ba98-c261119e42ab admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:46.347235 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:46.366218 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c2a3d105-4625-4410-9005-bfc1eb1c9580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5693cbae8eb4f34a7a7b216223a3d1f', 'request_id': 'req-c2a3d105-4625-4410-9005-bfc1eb1c9580'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:46.432499 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 826/1666] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:46 2022] POST /identity/v3/projects => generated 365 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:46.434361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3403b-3c12-4b63-ba98-c261119e42ab admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:46.512173 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c2a3d105-4625-4410-9005-bfc1eb1c9580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5693cbae8eb4f34a7a7b216223a3d1f', 'request_id': 'req-c2a3d105-4625-4410-9005-bfc1eb1c9580'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:46.521244 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c2a3d105-4625-4410-9005-bfc1eb1c9580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5693cbae8eb4f34a7a7b216223a3d1f', 'request_id': 'req-c2a3d105-4625-4410-9005-bfc1eb1c9580'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:46.566179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c2a3d105-4625-4410-9005-bfc1eb1c9580 admin admin] The token cache is being invalidate because project b5693cbae8eb4f34a7a7b216223a3d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:59:46.566656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c2a3d105-4625-4410-9005-bfc1eb1c9580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:46.567719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c2a3d105-4625-4410-9005-bfc1eb1c9580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:46.571508 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 841/1667] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 13:59:45 2022] DELETE /identity/v3/projects/b5693cbae8eb4f34a7a7b216223a3d1f => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:46.576854 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3d105-4625-4410-9005-bfc1eb1c9580 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:46.582852 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c2a3d105-4625-4410-9005-bfc1eb1c9580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:46.593790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3403b-3c12-4b63-ba98-c261119e42ab admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:46.607980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d23f13-9ffa-4e6c-92e5-76db57ee4d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:46.609247 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d23f13-9ffa-4e6c-92e5-76db57ee4d7b admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:46.609606 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d23f13-9ffa-4e6c-92e5-76db57ee4d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:46.609945 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d23f13-9ffa-4e6c-92e5-76db57ee4d7b admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:46.611618 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d23f13-9ffa-4e6c-92e5-76db57ee4d7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:46.621324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d23f13-9ffa-4e6c-92e5-76db57ee4d7b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:46.733610 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3d105-4625-4410-9005-bfc1eb1c9580 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:46.741278 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203f00c-062f-43c8-a05f-41aa86ee3fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:46.746434 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203f00c-062f-43c8-a05f-41aa86ee3fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:46.747634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203f00c-062f-43c8-a05f-41aa86ee3fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:46.748505 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203f00c-062f-43c8-a05f-41aa86ee3fa4 admin admin] PATH_INFO: `/v3/users/5ad3d83a74ef4472acb2422658775c66` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:46.750308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d203f00c-062f-43c8-a05f-41aa86ee3fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ad3d83a74ef4472acb2422658775c66)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:46.753881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d203f00c-062f-43c8-a05f-41aa86ee3fa4 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:46.759309 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 827/1668] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:46 2022] POST /identity/v3/users => generated 283 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:46.763907 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d23f13-9ffa-4e6c-92e5-76db57ee4d7b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:46.771971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c9d23f13-9ffa-4e6c-92e5-76db57ee4d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:46.809266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d23f13-9ffa-4e6c-92e5-76db57ee4d7b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:46.828119 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1955f5f-dd53-4041-9ffa-8d726b6e50fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:46.828753 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1955f5f-dd53-4041-9ffa-8d726b6e50fe admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:46.829029 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1955f5f-dd53-4041-9ffa-8d726b6e50fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:46.829466 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1955f5f-dd53-4041-9ffa-8d726b6e50fe admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:46.830252 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1955f5f-dd53-4041-9ffa-8d726b6e50fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:46.832245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1955f5f-dd53-4041-9ffa-8d726b6e50fe admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:46.865723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 828/1669] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:46.867219 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1955f5f-dd53-4041-9ffa-8d726b6e50fe admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:46.894803 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1955f5f-dd53-4041-9ffa-8d726b6e50fe admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:46.914027 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a11fd-3475-464a-aa45-d0da5a7ee55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:46.914865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a11fd-3475-464a-aa45-d0da5a7ee55e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:46.915319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a11fd-3475-464a-aa45-d0da5a7ee55e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:46.915732 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a11fd-3475-464a-aa45-d0da5a7ee55e admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:46.916531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a11fd-3475-464a-aa45-d0da5a7ee55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:46.918487 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a11fd-3475-464a-aa45-d0da5a7ee55e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:46.947801 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 829/1670] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:46.952705 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a11fd-3475-464a-aa45-d0da5a7ee55e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:46.969733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a11fd-3475-464a-aa45-d0da5a7ee55e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:46.984389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5412f-8ce0-440f-8e15-719337288aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:46.985385 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5412f-8ce0-440f-8e15-719337288aec admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:46.985806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5412f-8ce0-440f-8e15-719337288aec admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:46.986369 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5412f-8ce0-440f-8e15-719337288aec admin admin] PATH_INFO: `/v3/projects/c23226926d16424da3990da0aec21155/users/87a0afc07ce34a2ab7b094f82c14277d/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:47.027361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5412f-8ce0-440f-8e15-719337288aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=c23226926d16424da3990da0aec21155, user_id=87a0afc07ce34a2ab7b094f82c14277d, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:47.030554 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5412f-8ce0-440f-8e15-719337288aec admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:47.081599 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d203f00c-062f-43c8-a05f-41aa86ee3fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ad3d83a74ef4472acb2422658775c66'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:47.192960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 830/1671] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:46 2022] PUT /identity/v3/projects/c23226926d16424da3990da0aec21155/users/87a0afc07ce34a2ab7b094f82c14277d/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:59:47.198563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09162f21-f1ef-4aff-985b-b8aebbeba8c2 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:47.199339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09162f21-f1ef-4aff-985b-b8aebbeba8c2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:47.199758 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09162f21-f1ef-4aff-985b-b8aebbeba8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:47.203798 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d203f00c-062f-43c8-a05f-41aa86ee3fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ad3d83a74ef4472acb2422658775c66'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:47.204753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d203f00c-062f-43c8-a05f-41aa86ee3fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ad3d83a74ef4472acb2422658775c66'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:47.215939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d203f00c-062f-43c8-a05f-41aa86ee3fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ad3d83a74ef4472acb2422658775c66'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:47.241854 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d203f00c-062f-43c8-a05f-41aa86ee3fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ad3d83a74ef4472acb2422658775c66'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:47.261304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 842/1672] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 13:59:46 2022] DELETE /identity/v3/users/5ad3d83a74ef4472acb2422658775c66 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:47.297396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-09162f21-f1ef-4aff-985b-b8aebbeba8c2 None None] MFA Rules not processed for user `87a0afc07ce34a2ab7b094f82c14277d`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:47.300421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-09162f21-f1ef-4aff-985b-b8aebbeba8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:47.437411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 831/1673] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:47.442850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203f00c-062f-43c8-a05f-41aa86ee3fa4 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:47.444455 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d203f00c-062f-43c8-a05f-41aa86ee3fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:47.585847 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203f00c-062f-43c8-a05f-41aa86ee3fa4 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:47.603698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0be12-84b2-46c8-84a0-b25db100402c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:47.604497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0be12-84b2-46c8-84a0-b25db100402c admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:47.605570 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0be12-84b2-46c8-84a0-b25db100402c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:47.606045 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0be12-84b2-46c8-84a0-b25db100402c admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:47.606917 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0be12-84b2-46c8-84a0-b25db100402c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:47.609282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f0be12-84b2-46c8-84a0-b25db100402c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:47.646345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:47.648288 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09162f21-f1ef-4aff-985b-b8aebbeba8c2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:47.652197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-09162f21-f1ef-4aff-985b-b8aebbeba8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:47.680860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09162f21-f1ef-4aff-985b-b8aebbeba8c2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:47.714946 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce93110-c339-4dc8-9e92-45c0f30a603b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:47.715627 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce93110-c339-4dc8-9e92-45c0f30a603b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:47.715908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce93110-c339-4dc8-9e92-45c0f30a603b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:47.716175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce93110-c339-4dc8-9e92-45c0f30a603b admin admin] PATH_INFO: `/v3/projects/266ca24a1fb44f9992e9eb372695bdc6` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:47.722313 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce93110-c339-4dc8-9e92-45c0f30a603b admin admin] RBAC: Authorizing `identity:delete_project(project_id=266ca24a1fb44f9992e9eb372695bdc6)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:47.724326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce93110-c339-4dc8-9e92-45c0f30a603b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:47.859501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 843/1674] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:47 2022] POST /identity/v3/projects => generated 365 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:47.863706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0be12-84b2-46c8-84a0-b25db100402c admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:47.906824 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:47.959590 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4ce93110-c339-4dc8-9e92-45c0f30a603b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '266ca24a1fb44f9992e9eb372695bdc6', 'request_id': 'req-4ce93110-c339-4dc8-9e92-45c0f30a603b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:47.980127 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4ce93110-c339-4dc8-9e92-45c0f30a603b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '266ca24a1fb44f9992e9eb372695bdc6', 'request_id': 'req-4ce93110-c339-4dc8-9e92-45c0f30a603b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:47.991194 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4ce93110-c339-4dc8-9e92-45c0f30a603b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '266ca24a1fb44f9992e9eb372695bdc6', 'request_id': 'req-4ce93110-c339-4dc8-9e92-45c0f30a603b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:47.998175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0be12-84b2-46c8-84a0-b25db100402c admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:48.147029 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4ce93110-c339-4dc8-9e92-45c0f30a603b admin admin] The token cache is being invalidate because project 266ca24a1fb44f9992e9eb372695bdc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:59:48.147430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4ce93110-c339-4dc8-9e92-45c0f30a603b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:48.149576 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4ce93110-c339-4dc8-9e92-45c0f30a603b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:48.153559 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 832/1675] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 13:59:47 2022] DELETE /identity/v3/projects/266ca24a1fb44f9992e9eb372695bdc6 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:48.158205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e374948-c134-4c92-8e7f-a8ae8e175c1c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:48.158673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e374948-c134-4c92-8e7f-a8ae8e175c1c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:48.159251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e374948-c134-4c92-8e7f-a8ae8e175c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:48.197577 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-2e374948-c134-4c92-8e7f-a8ae8e175c1c None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:48.199522 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2e374948-c134-4c92-8e7f-a8ae8e175c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:48.260320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da750cee-00a2-40e0-bbc1-281902769171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:48.265970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da750cee-00a2-40e0-bbc1-281902769171 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:48.265970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da750cee-00a2-40e0-bbc1-281902769171 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:48.265970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da750cee-00a2-40e0-bbc1-281902769171 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:48.266388 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da750cee-00a2-40e0-bbc1-281902769171 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:48.272281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da750cee-00a2-40e0-bbc1-281902769171 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:48.352179 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 833/1676] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:59:48.361641 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955e393-fd57-4172-8369-a75659a9eb4e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:48.362102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955e393-fd57-4172-8369-a75659a9eb4e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:48.362475 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955e393-fd57-4172-8369-a75659a9eb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:48.409885 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8955e393-fd57-4172-8369-a75659a9eb4e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:48.411966 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8955e393-fd57-4172-8369-a75659a9eb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:48.428351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 844/1677] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:47 2022] POST /identity/v3/users => generated 289 bytes in 565 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:48.429486 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 834/1678] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:48.433670 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da750cee-00a2-40e0-bbc1-281902769171 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:48.435369 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955e393-fd57-4172-8369-a75659a9eb4e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:48.439464 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-da750cee-00a2-40e0-bbc1-281902769171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:48.465931 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955e393-fd57-4172-8369-a75659a9eb4e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:48.468778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da750cee-00a2-40e0-bbc1-281902769171 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:48.483693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac511d-5373-4541-b43e-79a6327086cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:48.485074 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac511d-5373-4541-b43e-79a6327086cf admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:48.486103 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b39603-44ce-4072-a021-f41e200fc82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:48.486664 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac511d-5373-4541-b43e-79a6327086cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:48.487565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b39603-44ce-4072-a021-f41e200fc82b admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:48.493401 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac511d-5373-4541-b43e-79a6327086cf admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:48.493401 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac511d-5373-4541-b43e-79a6327086cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:59:48.493401 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac511d-5373-4541-b43e-79a6327086cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:48.493767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b39603-44ce-4072-a021-f41e200fc82b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:48.493767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b39603-44ce-4072-a021-f41e200fc82b admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:48.493767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b39603-44ce-4072-a021-f41e200fc82b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:48.495136 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac511d-5373-4541-b43e-79a6327086cf admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:48.496970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b39603-44ce-4072-a021-f41e200fc82b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:48.523841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 845/1679] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:48.527584 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b39603-44ce-4072-a021-f41e200fc82b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:48.532001 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 835/1680] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 13:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:59:48.536846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac511d-5373-4541-b43e-79a6327086cf admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:48.554792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b39603-44ce-4072-a021-f41e200fc82b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:48.565097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac511d-5373-4541-b43e-79a6327086cf admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:48.575809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a6d72-b7cf-4c4c-a7fe-6755e473b280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:48.578851 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a6d72-b7cf-4c4c-a7fe-6755e473b280 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:48.580131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a6d72-b7cf-4c4c-a7fe-6755e473b280 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:48.580828 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a6d72-b7cf-4c4c-a7fe-6755e473b280 admin admin] PATH_INFO: `/v3/projects/66bdf3e7b1ed4565a08bbcaea3bc1ec8/users/8de2aa3c19f649ea823c8d99ba1ffdbb/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:48.585323 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15701db4-941a-4795-a892-67baf90c52db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:48.589203 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15701db4-941a-4795-a892-67baf90c52db admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:48.591293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15701db4-941a-4795-a892-67baf90c52db admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:48.591609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15701db4-941a-4795-a892-67baf90c52db admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:48.594072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15701db4-941a-4795-a892-67baf90c52db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:48.598721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15701db4-941a-4795-a892-67baf90c52db admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:48.633175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a6d72-b7cf-4c4c-a7fe-6755e473b280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bdf3e7b1ed4565a08bbcaea3bc1ec8, user_id=8de2aa3c19f649ea823c8d99ba1ffdbb, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:48.644549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a6d72-b7cf-4c4c-a7fe-6755e473b280 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:48.786918 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 846/1681] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:48 2022] PUT /identity/v3/projects/66bdf3e7b1ed4565a08bbcaea3bc1ec8/users/8de2aa3c19f649ea823c8d99ba1ffdbb/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:48.787889 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a6d72-b7cf-4c4c-a7fe-6755e473b280 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:48.791420 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 836/1682] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:48 2022] POST /identity/v3/projects => generated 365 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:48.793895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c93a6d72-b7cf-4c4c-a7fe-6755e473b280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:48.795809 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15701db4-941a-4795-a892-67baf90c52db admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:48.937864 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15701db4-941a-4795-a892-67baf90c52db admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:48.943095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52022d6b-2823-4412-a75b-970821bdd5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:48.943940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52022d6b-2823-4412-a75b-970821bdd5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:48.944776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52022d6b-2823-4412-a75b-970821bdd5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:48.944776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52022d6b-2823-4412-a75b-970821bdd5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:48.946796 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52022d6b-2823-4412-a75b-970821bdd5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:48.947927 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52022d6b-2823-4412-a75b-970821bdd5c4 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:48.954809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a6d72-b7cf-4c4c-a7fe-6755e473b280 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:48.962218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 837/1683] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:48.963341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52022d6b-2823-4412-a75b-970821bdd5c4 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:48.967784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3063cc-6b35-400f-a5de-e0cfa778b58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:48.968627 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3063cc-6b35-400f-a5de-e0cfa778b58c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:48.969055 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3063cc-6b35-400f-a5de-e0cfa778b58c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:48.969475 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3063cc-6b35-400f-a5de-e0cfa778b58c admin admin] PATH_INFO: `/v3/projects/624feb8c32d24b0781cc585c797c01e5` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:48.970846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3063cc-6b35-400f-a5de-e0cfa778b58c admin admin] RBAC: Authorizing `identity:delete_project(project_id=624feb8c32d24b0781cc585c797c01e5)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:48.979059 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3063cc-6b35-400f-a5de-e0cfa778b58c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:48.979727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52022d6b-2823-4412-a75b-970821bdd5c4 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:48.987229 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49f9ed-6d19-44f3-ac75-31710c4c4a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:48.987986 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49f9ed-6d19-44f3-ac75-31710c4c4a86 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:48.988333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49f9ed-6d19-44f3-ac75-31710c4c4a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:48.989081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49f9ed-6d19-44f3-ac75-31710c4c4a86 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:48.989890 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49f9ed-6d19-44f3-ac75-31710c4c4a86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:48.992094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49f9ed-6d19-44f3-ac75-31710c4c4a86 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:49.191462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 838/1684] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:48 2022] POST /identity/v3/users => generated 283 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:49.193425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49f9ed-6d19-44f3-ac75-31710c4c4a86 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:49.209704 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e3063cc-6b35-400f-a5de-e0cfa778b58c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '624feb8c32d24b0781cc585c797c01e5', 'request_id': 'req-2e3063cc-6b35-400f-a5de-e0cfa778b58c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:49.238359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49f9ed-6d19-44f3-ac75-31710c4c4a86 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:49.342281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e3063cc-6b35-400f-a5de-e0cfa778b58c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '624feb8c32d24b0781cc585c797c01e5', 'request_id': 'req-2e3063cc-6b35-400f-a5de-e0cfa778b58c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:49.354414 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e3063cc-6b35-400f-a5de-e0cfa778b58c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '624feb8c32d24b0781cc585c797c01e5', 'request_id': 'req-2e3063cc-6b35-400f-a5de-e0cfa778b58c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:49.366152 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38988d66-2688-4a93-90bd-80b257f23ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:49.369165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38988d66-2688-4a93-90bd-80b257f23ade admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:49.369460 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38988d66-2688-4a93-90bd-80b257f23ade admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:49.369736 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38988d66-2688-4a93-90bd-80b257f23ade admin admin] PATH_INFO: `/v3/projects/66bdf3e7b1ed4565a08bbcaea3bc1ec8/users/8de2aa3c19f649ea823c8d99ba1ffdbb/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:49.375467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38988d66-2688-4a93-90bd-80b257f23ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bdf3e7b1ed4565a08bbcaea3bc1ec8, user_id=8de2aa3c19f649ea823c8d99ba1ffdbb, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:49.384970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38988d66-2688-4a93-90bd-80b257f23ade admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:49.396187 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e3063cc-6b35-400f-a5de-e0cfa778b58c admin admin] The token cache is being invalidate because project 624feb8c32d24b0781cc585c797c01e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:59:49.396187 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e3063cc-6b35-400f-a5de-e0cfa778b58c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:49.407394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e3063cc-6b35-400f-a5de-e0cfa778b58c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:49.409950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 847/1685] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 13:59:48 2022] DELETE /identity/v3/projects/624feb8c32d24b0781cc585c797c01e5 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:49.411112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3063cc-6b35-400f-a5de-e0cfa778b58c admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:49.413426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2e3063cc-6b35-400f-a5de-e0cfa778b58c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:49.429969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 839/1686] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:49 2022] PUT /identity/v3/projects/66bdf3e7b1ed4565a08bbcaea3bc1ec8/users/8de2aa3c19f649ea823c8d99ba1ffdbb/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:49.432098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b054ec9-2698-4f7d-b2de-5d08896c7797 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:49.433365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b054ec9-2698-4f7d-b2de-5d08896c7797 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:49.433972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b054ec9-2698-4f7d-b2de-5d08896c7797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:49.494439 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3063cc-6b35-400f-a5de-e0cfa778b58c admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:49.570506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8b054ec9-2698-4f7d-b2de-5d08896c7797 None None] MFA Rules not processed for user `a8aafe3997c144c08edd2b829ae51135`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:49.571790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8b054ec9-2698-4f7d-b2de-5d08896c7797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:49.686502 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 840/1687] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:49 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:49.688633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0559365-b5ca-4c78-bad8-54d93476cf06 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:49.688926 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0559365-b5ca-4c78-bad8-54d93476cf06 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:49.689207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0559365-b5ca-4c78-bad8-54d93476cf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:49.694057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b35c3-4951-43a5-a52b-56dff8371f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:49.696128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b35c3-4951-43a5-a52b-56dff8371f74 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:49.696724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b35c3-4951-43a5-a52b-56dff8371f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:49.699265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b35c3-4951-43a5-a52b-56dff8371f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:49.700179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b35c3-4951-43a5-a52b-56dff8371f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:49.708067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b35c3-4951-43a5-a52b-56dff8371f74 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:49.719402 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b0559365-b5ca-4c78-bad8-54d93476cf06 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:49.720822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b0559365-b5ca-4c78-bad8-54d93476cf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:49.732479 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 848/1688] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:49.734372 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c881d-8ea8-4b44-b894-8428b64beb19 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:49.734792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c881d-8ea8-4b44-b894-8428b64beb19 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:49.735233 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c881d-8ea8-4b44-b894-8428b64beb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:49.752960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 841/1689] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:49.752960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0559365-b5ca-4c78-bad8-54d93476cf06 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:49.763350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0559365-b5ca-4c78-bad8-54d93476cf06 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:49.772122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ab257-7d24-4554-91ee-123efdba7780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:49.772935 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ab257-7d24-4554-91ee-123efdba7780 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:49.773220 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ab257-7d24-4554-91ee-123efdba7780 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:49.773489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ab257-7d24-4554-91ee-123efdba7780 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:49.782748 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ab257-7d24-4554-91ee-123efdba7780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:49.789072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ab257-7d24-4554-91ee-123efdba7780 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:49.818849 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-571c881d-8ea8-4b44-b894-8428b64beb19 None None] MFA Rules not processed for user `8de2aa3c19f649ea823c8d99ba1ffdbb`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:49.820646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-571c881d-8ea8-4b44-b894-8428b64beb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:49.842002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 842/1690] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:49.844432 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47990d-c5dc-48a8-9a11-3bb3d84ad837 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:49.844911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47990d-c5dc-48a8-9a11-3bb3d84ad837 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:49.845168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47990d-c5dc-48a8-9a11-3bb3d84ad837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:49.888557 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-cd47990d-c5dc-48a8-9a11-3bb3d84ad837 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:49.889887 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-cd47990d-c5dc-48a8-9a11-3bb3d84ad837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:49.915623 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 843/1691] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:49.916070 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47990d-c5dc-48a8-9a11-3bb3d84ad837 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:49.941306 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47990d-c5dc-48a8-9a11-3bb3d84ad837 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:49.971544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cc656-f498-424a-a4d7-8976a92a213f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:49.972538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cc656-f498-424a-a4d7-8976a92a213f admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:49.973415 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cc656-f498-424a-a4d7-8976a92a213f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:49.976117 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cc656-f498-424a-a4d7-8976a92a213f admin admin] PATH_INFO: `/v3/projects/adb7737a46da4ad6aa1dfc1c7974f346/users/2c6492631396418283e4ff1a1ae6289b/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:49.999503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:50.000442 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 849/1692] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:50.001521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c881d-8ea8-4b44-b894-8428b64beb19 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:50.022376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060cc656-f498-424a-a4d7-8976a92a213f admin admin] RBAC: Authorizing `identity:create_grant(project_id=adb7737a46da4ad6aa1dfc1c7974f346, user_id=2c6492631396418283e4ff1a1ae6289b, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:50.027222 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c881d-8ea8-4b44-b894-8428b64beb19 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:50.031898 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060cc656-f498-424a-a4d7-8976a92a213f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:50.043526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d49d6-bcfb-4cf8-87d4-23e6140e661e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:50.046659 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d49d6-bcfb-4cf8-87d4-23e6140e661e admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:50.046982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d49d6-bcfb-4cf8-87d4-23e6140e661e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:50.047690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d49d6-bcfb-4cf8-87d4-23e6140e661e admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:50.050228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d49d6-bcfb-4cf8-87d4-23e6140e661e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:59:50.050806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d49d6-bcfb-4cf8-87d4-23e6140e661e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:50.053972 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d49d6-bcfb-4cf8-87d4-23e6140e661e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:50.072552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:50.082438 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 850/1693] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 13:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:59:50.083639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d49d6-bcfb-4cf8-87d4-23e6140e661e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:50.085654 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-211d49d6-bcfb-4cf8-87d4-23e6140e661e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:50.165932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 844/1694] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:49 2022] PUT /identity/v3/projects/adb7737a46da4ad6aa1dfc1c7974f346/users/2c6492631396418283e4ff1a1ae6289b/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:50.168151 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cc656-f498-424a-a4d7-8976a92a213f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:50.257908 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d49d6-bcfb-4cf8-87d4-23e6140e661e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:50.366796 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cc656-f498-424a-a4d7-8976a92a213f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:50.379901 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef054e-33d3-43a0-9b9e-ba92459f272e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:50.383156 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef054e-33d3-43a0-9b9e-ba92459f272e admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:50.383156 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef054e-33d3-43a0-9b9e-ba92459f272e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:50.383472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef054e-33d3-43a0-9b9e-ba92459f272e admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:50.383941 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef054e-33d3-43a0-9b9e-ba92459f272e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:50.386020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef054e-33d3-43a0-9b9e-ba92459f272e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:50.418783 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c731cf6-b81b-4885-b177-da5f50254099 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:50.420235 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c731cf6-b81b-4885-b177-da5f50254099 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:50.420707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c731cf6-b81b-4885-b177-da5f50254099 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:50.421182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c731cf6-b81b-4885-b177-da5f50254099 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:50.545845 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c731cf6-b81b-4885-b177-da5f50254099 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:50.548124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c731cf6-b81b-4885-b177-da5f50254099 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:50.564326 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 851/1695] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:50 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:50.568335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301e183-f2df-41f5-a2ce-ff5d92da30d2 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:50.568777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301e183-f2df-41f5-a2ce-ff5d92da30d2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:50.569205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301e183-f2df-41f5-a2ce-ff5d92da30d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:50.604642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 845/1696] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:50 2022] POST /identity/v3/projects => generated 409 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:50.604642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b008e1c-8078-4a44-bf30-6c3ba264579d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:50.604642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b008e1c-8078-4a44-bf30-6c3ba264579d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:50.604642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b008e1c-8078-4a44-bf30-6c3ba264579d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:50.619885 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f301e183-f2df-41f5-a2ce-ff5d92da30d2 None None] MFA Rules not processed for user `2c6492631396418283e4ff1a1ae6289b`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:50.621410 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f301e183-f2df-41f5-a2ce-ff5d92da30d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:50.684637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0b008e1c-8078-4a44-bf30-6c3ba264579d None None] MFA Rules not processed for user `7b403bdc9cad4783b72bbf174199363c`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:50.685893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0b008e1c-8078-4a44-bf30-6c3ba264579d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:50.777039 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 852/1697] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:50.777970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301e183-f2df-41f5-a2ce-ff5d92da30d2 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:50.815227 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 846/1698] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:50.816471 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b008e1c-8078-4a44-bf30-6c3ba264579d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:50.969285 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301e183-f2df-41f5-a2ce-ff5d92da30d2 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:50.987821 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700bf8f-be25-4ea1-a48d-a3652907f48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:50.988072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b008e1c-8078-4a44-bf30-6c3ba264579d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:50.989333 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700bf8f-be25-4ea1-a48d-a3652907f48a admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:50.989618 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700bf8f-be25-4ea1-a48d-a3652907f48a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:50.989894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700bf8f-be25-4ea1-a48d-a3652907f48a admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:50.992768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700bf8f-be25-4ea1-a48d-a3652907f48a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:50.995051 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700bf8f-be25-4ea1-a48d-a3652907f48a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:51.000721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90854822-f5c0-4192-8b3c-4514d2472b00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:51.001418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90854822-f5c0-4192-8b3c-4514d2472b00 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:51.001706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90854822-f5c0-4192-8b3c-4514d2472b00 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:51.001998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90854822-f5c0-4192-8b3c-4514d2472b00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:51.026170 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90854822-f5c0-4192-8b3c-4514d2472b00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:51.028789 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90854822-f5c0-4192-8b3c-4514d2472b00 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:51.062843 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 847/1699] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:51.063699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90854822-f5c0-4192-8b3c-4514d2472b00 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:51.065451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-90854822-f5c0-4192-8b3c-4514d2472b00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:51.129761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 853/1700] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:50 2022] POST /identity/v3/users => generated 305 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:51.134606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700bf8f-be25-4ea1-a48d-a3652907f48a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:51.157236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700bf8f-be25-4ea1-a48d-a3652907f48a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:51.169641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f797b1-a977-44a1-b5c1-5ddefd5d0c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:51.170833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f797b1-a977-44a1-b5c1-5ddefd5d0c92 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:51.171146 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f797b1-a977-44a1-b5c1-5ddefd5d0c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:51.172323 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f797b1-a977-44a1-b5c1-5ddefd5d0c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:51.173054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f797b1-a977-44a1-b5c1-5ddefd5d0c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:51.177033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f797b1-a977-44a1-b5c1-5ddefd5d0c92 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:51.200046 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 854/1701] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:51.207008 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f797b1-a977-44a1-b5c1-5ddefd5d0c92 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:51.231018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f797b1-a977-44a1-b5c1-5ddefd5d0c92 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:51.259480 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb7f6d-81f5-4d3d-ae8c-f7da8beea54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:51.260915 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb7f6d-81f5-4d3d-ae8c-f7da8beea54a admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:51.267585 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb7f6d-81f5-4d3d-ae8c-f7da8beea54a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:51.268793 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb7f6d-81f5-4d3d-ae8c-f7da8beea54a admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:51.269661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb7f6d-81f5-4d3d-ae8c-f7da8beea54a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:51.274153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:51.276052 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90854822-f5c0-4192-8b3c-4514d2472b00 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:51.277471 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb7f6d-81f5-4d3d-ae8c-f7da8beea54a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:51.298345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f87dad-d54e-422d-aa4c-43be7796ddb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:51.299095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f87dad-d54e-422d-aa4c-43be7796ddb0 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:51.301116 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f87dad-d54e-422d-aa4c-43be7796ddb0 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:51.301508 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f87dad-d54e-422d-aa4c-43be7796ddb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:51.317538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 855/1702] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:51.320909 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb7f6d-81f5-4d3d-ae8c-f7da8beea54a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:51.333714 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f87dad-d54e-422d-aa4c-43be7796ddb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:51.340954 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f87dad-d54e-422d-aa4c-43be7796ddb0 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:51.363875 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb7f6d-81f5-4d3d-ae8c-f7da8beea54a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:51.384492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 848/1703] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:51.397644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fd4c1-3681-4010-b59f-e428a1d73448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:51.398471 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fd4c1-3681-4010-b59f-e428a1d73448 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:51.398825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fd4c1-3681-4010-b59f-e428a1d73448 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:51.399218 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fd4c1-3681-4010-b59f-e428a1d73448 admin admin] PATH_INFO: `/v3/projects/f01d60ca831b4b2ab0b1fdea2cee9696/users/9913db0bd00f4560ae6c595b35bc92ec/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:51.438222 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fd4c1-3681-4010-b59f-e428a1d73448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f01d60ca831b4b2ab0b1fdea2cee9696, user_id=9913db0bd00f4560ae6c595b35bc92ec, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:51.441441 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fd4c1-3681-4010-b59f-e428a1d73448 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:51.584683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 856/1704] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:51 2022] PUT /identity/v3/projects/f01d60ca831b4b2ab0b1fdea2cee9696/users/9913db0bd00f4560ae6c595b35bc92ec/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:51.598250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56a3d9-8e4e-4cbf-b247-8daacd6d98f1 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:51.598618 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56a3d9-8e4e-4cbf-b247-8daacd6d98f1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:51.600404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56a3d9-8e4e-4cbf-b247-8daacd6d98f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:51.697633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7a56a3d9-8e4e-4cbf-b247-8daacd6d98f1 None None] MFA Rules not processed for user `9913db0bd00f4560ae6c595b35bc92ec`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:51.699134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7a56a3d9-8e4e-4cbf-b247-8daacd6d98f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:51.802474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 849/1705] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:51 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:51.806475 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fd4c1-3681-4010-b59f-e428a1d73448 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:51.938616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fd4c1-3681-4010-b59f-e428a1d73448 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:51.943670 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ff099-8443-42d4-afb3-49faf8512f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:51.946492 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ff099-8443-42d4-afb3-49faf8512f3c admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:51.956578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:51.956578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ff099-8443-42d4-afb3-49faf8512f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:51.956578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ff099-8443-42d4-afb3-49faf8512f3c admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:51.956578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ff099-8443-42d4-afb3-49faf8512f3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:51.956578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ff099-8443-42d4-afb3-49faf8512f3c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:52.001238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599b5ef-2131-454a-b9ce-1202bc87d268 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.001799 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599b5ef-2131-454a-b9ce-1202bc87d268 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.003484 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599b5ef-2131-454a-b9ce-1202bc87d268 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.004895 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 850/1706] 10.222.0.91 () {54 vars in 921 bytes} [Thu Sep 8 13:59:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:59:52.012631 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295070b5-2c33-49f4-83b1-a924f541166f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.012922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295070b5-2c33-49f4-83b1-a924f541166f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.013177 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295070b5-2c33-49f4-83b1-a924f541166f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.073951 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-295070b5-2c33-49f4-83b1-a924f541166f None None] MFA Rules not processed for user `f9506139d5744fe69153e7e30a05b436`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:52.076391 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-295070b5-2c33-49f4-83b1-a924f541166f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:52.130545 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 857/1707] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:51 2022] POST /identity/v3/projects => generated 407 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:52.134633 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ff099-8443-42d4-afb3-49faf8512f3c admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:52.247051 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 851/1708] 10.222.0.91 () {58 vars in 1018 bytes} [Thu Sep 8 13:59:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:52.294201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295070b5-2c33-49f4-83b1-a924f541166f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:52.295713 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-295070b5-2c33-49f4-83b1-a924f541166f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:52.321918 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ff099-8443-42d4-afb3-49faf8512f3c admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:52.331634 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295070b5-2c33-49f4-83b1-a924f541166f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:52.350400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5565a77-c150-4818-8f9e-7fb96e2565cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:52.353511 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5565a77-c150-4818-8f9e-7fb96e2565cb admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.354148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5565a77-c150-4818-8f9e-7fb96e2565cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.354148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5565a77-c150-4818-8f9e-7fb96e2565cb admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.358399 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5565a77-c150-4818-8f9e-7fb96e2565cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:52.365117 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5565a77-c150-4818-8f9e-7fb96e2565cb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:52.498928 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9e4a2-d998-44e7-81bf-e9c1c7bb7beb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9506139d5744fe69153e7e30a05b436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:52.499757 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9e4a2-d998-44e7-81bf-e9c1c7bb7beb service placement] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.499862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9e4a2-d998-44e7-81bf-e9c1c7bb7beb service placement] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.500248 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9e4a2-d998-44e7-81bf-e9c1c7bb7beb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.522225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9e4a2-d998-44e7-81bf-e9c1c7bb7beb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:52.523916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9e4a2-d998-44e7-81bf-e9c1c7bb7beb service placement] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:52.537029 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 858/1709] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:52 2022] POST /identity/v3/users => generated 310 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:52.541674 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5565a77-c150-4818-8f9e-7fb96e2565cb admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:52.564114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5565a77-c150-4818-8f9e-7fb96e2565cb admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:52.570950 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 852/1710] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:52.586764 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54137973-0256-4580-b717-5e6f77819020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:52.587486 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54137973-0256-4580-b717-5e6f77819020 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.587761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54137973-0256-4580-b717-5e6f77819020 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.588038 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54137973-0256-4580-b717-5e6f77819020 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.590060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54137973-0256-4580-b717-5e6f77819020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:52.592128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54137973-0256-4580-b717-5e6f77819020 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:52.612123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 859/1711] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:59:52.616310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9e4a2-d998-44e7-81bf-e9c1c7bb7beb service placement] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:52.622490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54137973-0256-4580-b717-5e6f77819020 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:52.626434 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-54137973-0256-4580-b717-5e6f77819020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:52.637309 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9e4a2-d998-44e7-81bf-e9c1c7bb7beb service placement] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:52.642583 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bf0f4-7873-4b68-95aa-2096aeb0cadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:52.643745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bf0f4-7873-4b68-95aa-2096aeb0cadf admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.643745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bf0f4-7873-4b68-95aa-2096aeb0cadf admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.643909 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11bf0f4-7873-4b68-95aa-2096aeb0cadf admin admin] PATH_INFO: `/v3/projects/fe1541151785413590856ae0585ff8e1/users/d766ddfd9cb64363b667ed79096490d2/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.665458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54137973-0256-4580-b717-5e6f77819020 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:52.685307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ed4da-c72d-409b-9da2-b302eb4c0f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:52.685996 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ed4da-c72d-409b-9da2-b302eb4c0f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.689701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ed4da-c72d-409b-9da2-b302eb4c0f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.689701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ed4da-c72d-409b-9da2-b302eb4c0f65 admin admin] PATH_INFO: `/v3/users/87a0afc07ce34a2ab7b094f82c14277d` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.693351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11bf0f4-7873-4b68-95aa-2096aeb0cadf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1541151785413590856ae0585ff8e1, user_id=d766ddfd9cb64363b667ed79096490d2, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:52.696064 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11bf0f4-7873-4b68-95aa-2096aeb0cadf admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:52.711643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ed4da-c72d-409b-9da2-b302eb4c0f65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87a0afc07ce34a2ab7b094f82c14277d)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:52.719917 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ed4da-c72d-409b-9da2-b302eb4c0f65 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:52.870717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 853/1712] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:52 2022] PUT /identity/v3/projects/fe1541151785413590856ae0585ff8e1/users/d766ddfd9cb64363b667ed79096490d2/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:52.875539 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bf0f4-7873-4b68-95aa-2096aeb0cadf admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:52.970029 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11bf0f4-7873-4b68-95aa-2096aeb0cadf admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:52.976648 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f17e6-7f5b-40b6-9412-6274d8665c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:52.977541 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f17e6-7f5b-40b6-9412-6274d8665c3a admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.978192 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f17e6-7f5b-40b6-9412-6274d8665c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.978734 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f17e6-7f5b-40b6-9412-6274d8665c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.979765 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f17e6-7f5b-40b6-9412-6274d8665c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:52.982063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f17e6-7f5b-40b6-9412-6274d8665c3a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:52.994687 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 854/1713] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:52.999294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f17e6-7f5b-40b6-9412-6274d8665c3a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:53.048526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f17e6-7f5b-40b6-9412-6274d8665c3a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:53.053111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a75ed4da-c72d-409b-9da2-b302eb4c0f65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87a0afc07ce34a2ab7b094f82c14277d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:53.061646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efb857-fc04-4296-8117-86aae27f871b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:53.062597 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efb857-fc04-4296-8117-86aae27f871b admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:53.063058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efb857-fc04-4296-8117-86aae27f871b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:53.063266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efb857-fc04-4296-8117-86aae27f871b admin admin] PATH_INFO: `/v3/projects/fe1541151785413590856ae0585ff8e1/users/d766ddfd9cb64363b667ed79096490d2/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:53.065525 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efb857-fc04-4296-8117-86aae27f871b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1541151785413590856ae0585ff8e1, user_id=d766ddfd9cb64363b667ed79096490d2, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:53.068337 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efb857-fc04-4296-8117-86aae27f871b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:53.084089 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 855/1714] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:52 2022] PUT /identity/v3/projects/fe1541151785413590856ae0585ff8e1/users/d766ddfd9cb64363b667ed79096490d2/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:53.102911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d271be5-1efc-420f-a35e-02f40a5c1f82 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:53.103446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d271be5-1efc-420f-a35e-02f40a5c1f82 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:53.103964 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d271be5-1efc-420f-a35e-02f40a5c1f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:53.180796 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a75ed4da-c72d-409b-9da2-b302eb4c0f65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87a0afc07ce34a2ab7b094f82c14277d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:53.181756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a75ed4da-c72d-409b-9da2-b302eb4c0f65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87a0afc07ce34a2ab7b094f82c14277d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:53.197803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a75ed4da-c72d-409b-9da2-b302eb4c0f65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87a0afc07ce34a2ab7b094f82c14277d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:53.199872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7d271be5-1efc-420f-a35e-02f40a5c1f82 None None] MFA Rules not processed for user `d766ddfd9cb64363b667ed79096490d2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:53.201120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7d271be5-1efc-420f-a35e-02f40a5c1f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:53.236656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a75ed4da-c72d-409b-9da2-b302eb4c0f65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87a0afc07ce34a2ab7b094f82c14277d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:53.256966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 860/1715] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 13:59:52 2022] DELETE /identity/v3/users/87a0afc07ce34a2ab7b094f82c14277d => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:53.266811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a5e2e-8024-435f-82c3-a316b580b7e4 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:53.267123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a5e2e-8024-435f-82c3-a316b580b7e4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:53.267405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a5e2e-8024-435f-82c3-a316b580b7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:53.306495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-8c4a5e2e-8024-435f-82c3-a316b580b7e4 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:53.313069 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8c4a5e2e-8024-435f-82c3-a316b580b7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:53.393724 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 856/1716] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:53 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:53.404166 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d271be5-1efc-420f-a35e-02f40a5c1f82 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:53.405668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7d271be5-1efc-420f-a35e-02f40a5c1f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:53.497474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 861/1717] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:53.544065 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4a5e2e-8024-435f-82c3-a316b580b7e4 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:53.545886 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8c4a5e2e-8024-435f-82c3-a316b580b7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:53.563529 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d271be5-1efc-420f-a35e-02f40a5c1f82 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:53.678114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4a5e2e-8024-435f-82c3-a316b580b7e4 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:53.687488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb95245-fe4a-4e32-822c-10924184f562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:53.690054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb95245-fe4a-4e32-822c-10924184f562 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:53.690498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb95245-fe4a-4e32-822c-10924184f562 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:53.690779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb95245-fe4a-4e32-822c-10924184f562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:53.692015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40795287-fe46-42a0-97b6-e465f85634fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:53.692682 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40795287-fe46-42a0-97b6-e465f85634fc service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:53.692938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40795287-fe46-42a0-97b6-e465f85634fc service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:53.693230 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40795287-fe46-42a0-97b6-e465f85634fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:53.719007 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb95245-fe4a-4e32-822c-10924184f562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:53.722700 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb95245-fe4a-4e32-822c-10924184f562 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:53.741550 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 862/1718] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:53.866559 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40795287-fe46-42a0-97b6-e465f85634fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:53.868875 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40795287-fe46-42a0-97b6-e465f85634fc service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:53.883199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 857/1719] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:53 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:59:54.038240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb95245-fe4a-4e32-822c-10924184f562 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:54.041693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ceb95245-fe4a-4e32-822c-10924184f562 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:54.076044 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb95245-fe4a-4e32-822c-10924184f562 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:54.083432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d384b59-baa7-4b99-8551-3c7abd44d83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:54.084309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d384b59-baa7-4b99-8551-3c7abd44d83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:54.084665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d384b59-baa7-4b99-8551-3c7abd44d83f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:54.085047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d384b59-baa7-4b99-8551-3c7abd44d83f admin admin] PATH_INFO: `/v3/users/8161f28a4a1845c280cbe1c9f4cd674a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:54.086645 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d384b59-baa7-4b99-8551-3c7abd44d83f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8161f28a4a1845c280cbe1c9f4cd674a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:54.089038 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d384b59-baa7-4b99-8551-3c7abd44d83f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:54.319918 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:54.416578 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40795287-fe46-42a0-97b6-e465f85634fc service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:54.435948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40795287-fe46-42a0-97b6-e465f85634fc service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:54.440182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cc98c-807d-4e5c-907d-866ff4924758 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:54.440930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cc98c-807d-4e5c-907d-866ff4924758 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:54.441320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cc98c-807d-4e5c-907d-866ff4924758 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:54.442130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cc98c-807d-4e5c-907d-866ff4924758 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:54.444999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:54.446431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-330cc98c-807d-4e5c-907d-866ff4924758 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:54.516596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3d384b59-baa7-4b99-8551-3c7abd44d83f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8161f28a4a1845c280cbe1c9f4cd674a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:54.592977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cc98c-807d-4e5c-907d-866ff4924758 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:54.594454 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cc98c-807d-4e5c-907d-866ff4924758 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:54.605630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 858/1720] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:54 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:59:54.680114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3d384b59-baa7-4b99-8551-3c7abd44d83f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8161f28a4a1845c280cbe1c9f4cd674a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:54.683246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3d384b59-baa7-4b99-8551-3c7abd44d83f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8161f28a4a1845c280cbe1c9f4cd674a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:54.703686 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3d384b59-baa7-4b99-8551-3c7abd44d83f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8161f28a4a1845c280cbe1c9f4cd674a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:54.730702 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3d384b59-baa7-4b99-8551-3c7abd44d83f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8161f28a4a1845c280cbe1c9f4cd674a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:54.744015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 863/1721] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 13:59:54 2022] DELETE /identity/v3/users/8161f28a4a1845c280cbe1c9f4cd674a => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:55.263885 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cc98c-807d-4e5c-907d-866ff4924758 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:55.296227 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cc98c-807d-4e5c-907d-866ff4924758 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:55.447333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:55.450473 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:55.450955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:55.451460 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8 admin admin] PATH_INFO: `/v3/projects/5ba63eb7ead744f283bed215c7d25bda` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:55.453040 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba63eb7ead744f283bed215c7d25bda)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:55.455769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:55.797282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba63eb7ead744f283bed215c7d25bda', 'request_id': 'req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:55.814120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba63eb7ead744f283bed215c7d25bda', 'request_id': 'req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:55.842860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba63eb7ead744f283bed215c7d25bda', 'request_id': 'req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:56.020108 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8 admin admin] The token cache is being invalidate because project 5ba63eb7ead744f283bed215c7d25bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:59:56.020641 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:56.023544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:56.031759 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 859/1722] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 13:59:55 2022] DELETE /identity/v3/projects/5ba63eb7ead744f283bed215c7d25bda => generated 0 bytes in 769 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:56.038026 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d384b59-baa7-4b99-8551-3c7abd44d83f admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:56.040182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3d384b59-baa7-4b99-8551-3c7abd44d83f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:56.077979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d384b59-baa7-4b99-8551-3c7abd44d83f admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:56.355008 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15a852-f39e-481a-a80e-8114028993bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:56.363331 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15a852-f39e-481a-a80e-8114028993bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:56.363625 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15a852-f39e-481a-a80e-8114028993bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:56.364162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15a852-f39e-481a-a80e-8114028993bc admin admin] PATH_INFO: `/v3/users/a8aafe3997c144c08edd2b829ae51135` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:56.367789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15a852-f39e-481a-a80e-8114028993bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8aafe3997c144c08edd2b829ae51135)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:56.382711 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15a852-f39e-481a-a80e-8114028993bc admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:56.788433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5b15a852-f39e-481a-a80e-8114028993bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8aafe3997c144c08edd2b829ae51135'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:56.805481 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:56.812574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:56.836781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d4d41-65b2-4ce6-88c4-85f6d534ecc8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:56.856776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705484c9-37cb-47f6-aeb3-2680b8585f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:56.857710 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705484c9-37cb-47f6-aeb3-2680b8585f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:56.858474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705484c9-37cb-47f6-aeb3-2680b8585f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:56.859314 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705484c9-37cb-47f6-aeb3-2680b8585f60 admin admin] PATH_INFO: `/v3/projects/c23226926d16424da3990da0aec21155` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:56.860955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705484c9-37cb-47f6-aeb3-2680b8585f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c23226926d16424da3990da0aec21155)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:56.869509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705484c9-37cb-47f6-aeb3-2680b8585f60 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:56.923317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5b15a852-f39e-481a-a80e-8114028993bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8aafe3997c144c08edd2b829ae51135'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:56.924383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5b15a852-f39e-481a-a80e-8114028993bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8aafe3997c144c08edd2b829ae51135'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:56.944052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5b15a852-f39e-481a-a80e-8114028993bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8aafe3997c144c08edd2b829ae51135'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:56.972995 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5b15a852-f39e-481a-a80e-8114028993bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8aafe3997c144c08edd2b829ae51135'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:57.004551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 864/1723] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 13:59:56 2022] DELETE /identity/v3/users/a8aafe3997c144c08edd2b829ae51135 => generated 0 bytes in 962 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:57.168777 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-705484c9-37cb-47f6-aeb3-2680b8585f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c23226926d16424da3990da0aec21155', 'request_id': 'req-705484c9-37cb-47f6-aeb3-2680b8585f60'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:57.181508 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-705484c9-37cb-47f6-aeb3-2680b8585f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c23226926d16424da3990da0aec21155', 'request_id': 'req-705484c9-37cb-47f6-aeb3-2680b8585f60'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:57.230329 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-705484c9-37cb-47f6-aeb3-2680b8585f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c23226926d16424da3990da0aec21155', 'request_id': 'req-705484c9-37cb-47f6-aeb3-2680b8585f60'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:57.480682 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-705484c9-37cb-47f6-aeb3-2680b8585f60 admin admin] The token cache is being invalidate because project c23226926d16424da3990da0aec21155 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:59:57.483580 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-705484c9-37cb-47f6-aeb3-2680b8585f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:57.488893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-705484c9-37cb-47f6-aeb3-2680b8585f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:57.500571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15a852-f39e-481a-a80e-8114028993bc admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:57.501076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 860/1724] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 13:59:56 2022] DELETE /identity/v3/projects/c23226926d16424da3990da0aec21155 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:59:57.502832 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5b15a852-f39e-481a-a80e-8114028993bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:57.512562 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:57.673071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15a852-f39e-481a-a80e-8114028993bc admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:57.853694 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ee6c7-5199-469a-bcef-ded1e3f1f90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:57.854569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ee6c7-5199-469a-bcef-ded1e3f1f90d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:57.854857 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ee6c7-5199-469a-bcef-ded1e3f1f90d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:57.857202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ee6c7-5199-469a-bcef-ded1e3f1f90d admin admin] PATH_INFO: `/v3/users/8de2aa3c19f649ea823c8d99ba1ffdbb` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:57.858549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ee6c7-5199-469a-bcef-ded1e3f1f90d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8de2aa3c19f649ea823c8d99ba1ffdbb)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:57.860652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ee6c7-5199-469a-bcef-ded1e3f1f90d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:58.245599 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-893ee6c7-5199-469a-bcef-ded1e3f1f90d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8de2aa3c19f649ea823c8d99ba1ffdbb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:58.345283 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-893ee6c7-5199-469a-bcef-ded1e3f1f90d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8de2aa3c19f649ea823c8d99ba1ffdbb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:58.347478 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-893ee6c7-5199-469a-bcef-ded1e3f1f90d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8de2aa3c19f649ea823c8d99ba1ffdbb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:58.358778 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705484c9-37cb-47f6-aeb3-2680b8585f60 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:58.360750 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-705484c9-37cb-47f6-aeb3-2680b8585f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:58.388637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-893ee6c7-5199-469a-bcef-ded1e3f1f90d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8de2aa3c19f649ea823c8d99ba1ffdbb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:58.406540 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705484c9-37cb-47f6-aeb3-2680b8585f60 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:58.459719 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed778af1-2a7a-444a-9fea-cc8564f61116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:58.461985 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed778af1-2a7a-444a-9fea-cc8564f61116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:58.464220 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed778af1-2a7a-444a-9fea-cc8564f61116 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:58.465330 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed778af1-2a7a-444a-9fea-cc8564f61116 admin admin] PATH_INFO: `/v3/projects/516db22260e64e888a2c06164c83e3f0` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:58.471140 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed778af1-2a7a-444a-9fea-cc8564f61116 admin admin] RBAC: Authorizing `identity:delete_project(project_id=516db22260e64e888a2c06164c83e3f0)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:58.488830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-893ee6c7-5199-469a-bcef-ded1e3f1f90d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8de2aa3c19f649ea823c8d99ba1ffdbb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:58.489250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed778af1-2a7a-444a-9fea-cc8564f61116 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:58.535811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 865/1725] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 13:59:57 2022] DELETE /identity/v3/users/8de2aa3c19f649ea823c8d99ba1ffdbb => generated 0 bytes in 1036 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:59:58.711825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ed778af1-2a7a-444a-9fea-cc8564f61116 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '516db22260e64e888a2c06164c83e3f0', 'request_id': 'req-ed778af1-2a7a-444a-9fea-cc8564f61116'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:58.726522 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ed778af1-2a7a-444a-9fea-cc8564f61116 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '516db22260e64e888a2c06164c83e3f0', 'request_id': 'req-ed778af1-2a7a-444a-9fea-cc8564f61116'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:58.744410 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ed778af1-2a7a-444a-9fea-cc8564f61116 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '516db22260e64e888a2c06164c83e3f0', 'request_id': 'req-ed778af1-2a7a-444a-9fea-cc8564f61116'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:58.844244 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:58.853728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ed778af1-2a7a-444a-9fea-cc8564f61116 admin admin] The token cache is being invalidate because project 516db22260e64e888a2c06164c83e3f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:59:58.854108 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ed778af1-2a7a-444a-9fea-cc8564f61116 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:58.854787 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ed778af1-2a7a-444a-9fea-cc8564f61116 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:58.860675 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 861/1726] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 13:59:58 2022] DELETE /identity/v3/projects/516db22260e64e888a2c06164c83e3f0 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:58.866302 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9f6a2-3389-4731-88d3-e0ef97d32776 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:58.866608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9f6a2-3389-4731-88d3-e0ef97d32776 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:58.867502 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9f6a2-3389-4731-88d3-e0ef97d32776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:58.952154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-88a9f6a2-3389-4731-88d3-e0ef97d32776 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:58.958056 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-88a9f6a2-3389-4731-88d3-e0ef97d32776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:59.140856 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea4980-259b-46e1-9a3e-f2ce3425c6b8 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:59.141576 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea4980-259b-46e1-9a3e-f2ce3425c6b8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:59.142028 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea4980-259b-46e1-9a3e-f2ce3425c6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:59.142635 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 866/1727] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:59.176333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d4ea4980-259b-46e1-9a3e-f2ce3425c6b8 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:59.177760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d4ea4980-259b-46e1-9a3e-f2ce3425c6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:59.205127 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 862/1728] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 13:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:59:59.207401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9f6a2-3389-4731-88d3-e0ef97d32776 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:59.224195 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9f6a2-3389-4731-88d3-e0ef97d32776 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:59.236465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b29b3-9e0e-414e-ad2c-8a90a522507c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:59.237212 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b29b3-9e0e-414e-ad2c-8a90a522507c admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:59.240327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b29b3-9e0e-414e-ad2c-8a90a522507c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:59.240668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b29b3-9e0e-414e-ad2c-8a90a522507c admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:59.241466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b29b3-9e0e-414e-ad2c-8a90a522507c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:59:59.241836 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b29b3-9e0e-414e-ad2c-8a90a522507c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:59.243582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b29b3-9e0e-414e-ad2c-8a90a522507c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:59.272776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 867/1729] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 13:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:59:59.274310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea4980-259b-46e1-9a3e-f2ce3425c6b8 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:59.289159 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea4980-259b-46e1-9a3e-f2ce3425c6b8 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:59.303726 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995375d-6908-478e-b9eb-d30a7aae3f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:59.304365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995375d-6908-478e-b9eb-d30a7aae3f21 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:59.304829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995375d-6908-478e-b9eb-d30a7aae3f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:59.305098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8995375d-6908-478e-b9eb-d30a7aae3f21 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:59.305830 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8995375d-6908-478e-b9eb-d30a7aae3f21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:59.311215 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8995375d-6908-478e-b9eb-d30a7aae3f21 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:59.435008 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 863/1730] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:59 2022] POST /identity/v3/projects => generated 357 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:59:59.441331 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b29b3-9e0e-414e-ad2c-8a90a522507c admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:59.453249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b29b3-9e0e-414e-ad2c-8a90a522507c admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:59.520937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40ded9-615f-4476-a0ea-7d3650d8c5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:59.521654 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40ded9-615f-4476-a0ea-7d3650d8c5eb admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:59.521885 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40ded9-615f-4476-a0ea-7d3650d8c5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:59.522152 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40ded9-615f-4476-a0ea-7d3650d8c5eb admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:59.522782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e40ded9-615f-4476-a0ea-7d3650d8c5eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:59.533254 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e40ded9-615f-4476-a0ea-7d3650d8c5eb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:59.644951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 868/1731] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:59 2022] POST /identity/v3/users => generated 279 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:59.654367 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995375d-6908-478e-b9eb-d30a7aae3f21 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:59.697963 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8995375d-6908-478e-b9eb-d30a7aae3f21 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:59.718335 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56116150-bb7b-42d4-b967-737901c4221d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:59.719719 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56116150-bb7b-42d4-b967-737901c4221d admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:59.720852 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56116150-bb7b-42d4-b967-737901c4221d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:59.721542 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56116150-bb7b-42d4-b967-737901c4221d admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:59.722930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56116150-bb7b-42d4-b967-737901c4221d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:59.724613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:59.734181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56116150-bb7b-42d4-b967-737901c4221d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:59.747288 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:59.791385 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 864/1732] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:59.796396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40ded9-615f-4476-a0ea-7d3650d8c5eb admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:59.830753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40ded9-615f-4476-a0ea-7d3650d8c5eb admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:59.852468 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa04b0-9cd0-4dec-91bb-ca8682f71eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:59.853838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa04b0-9cd0-4dec-91bb-ca8682f71eff admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:59.856131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa04b0-9cd0-4dec-91bb-ca8682f71eff admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:59.857516 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa04b0-9cd0-4dec-91bb-ca8682f71eff admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:59.858682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa04b0-9cd0-4dec-91bb-ca8682f71eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:59.869710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa04b0-9cd0-4dec-91bb-ca8682f71eff admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:59.923954 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 869/1733] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:59.928821 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56116150-bb7b-42d4-b967-737901c4221d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:59.947427 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56116150-bb7b-42d4-b967-737901c4221d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:59.958359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d91ea3-aed9-4e03-a5c4-cf9d7c5f32bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:59.971865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d91ea3-aed9-4e03-a5c4-cf9d7c5f32bf admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:59.971865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d91ea3-aed9-4e03-a5c4-cf9d7c5f32bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:59.971865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d91ea3-aed9-4e03-a5c4-cf9d7c5f32bf admin admin] PATH_INFO: `/v3/projects/53f7dadb6ba6414298e5ebec52c6caa4/users/c0ba164b85cd4d0899a5f6c7f5b2b6fd/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:00.005239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d91ea3-aed9-4e03-a5c4-cf9d7c5f32bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=53f7dadb6ba6414298e5ebec52c6caa4, user_id=c0ba164b85cd4d0899a5f6c7f5b2b6fd, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:00.007311 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0ffc2-4c61-4ea9-a14c-65d461cb1fe8 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:00.007792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0ffc2-4c61-4ea9-a14c-65d461cb1fe8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:00.008259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0ffc2-4c61-4ea9-a14c-65d461cb1fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:00.009019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d91ea3-aed9-4e03-a5c4-cf9d7c5f32bf admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:00.112342 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 865/1734] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:59 2022] PUT /identity/v3/projects/53f7dadb6ba6414298e5ebec52c6caa4/users/c0ba164b85cd4d0899a5f6c7f5b2b6fd/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:00.118857 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8bcf1-7e38-4457-a4cf-5ebeb22978ce None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:00.122634 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8bcf1-7e38-4457-a4cf-5ebeb22978ce None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:00.123094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8bcf1-7e38-4457-a4cf-5ebeb22978ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:00.130605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-05d0ffc2-4c61-4ea9-a14c-65d461cb1fe8 None None] MFA Rules not processed for user `7b403bdc9cad4783b72bbf174199363c`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:00.154871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-05d0ffc2-4c61-4ea9-a14c-65d461cb1fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:00.202302 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e9d8bcf1-7e38-4457-a4cf-5ebeb22978ce None None] MFA Rules not processed for user `5771e9d0c5c94306b3ec571916acdcaa`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:00.204535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e9d8bcf1-7e38-4457-a4cf-5ebeb22978ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:00.326841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:00.379594 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 870/1735] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:00.381560 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c61b96-cdaa-48cc-9d02-38070ff2d7b9 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:00.381867 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c61b96-cdaa-48cc-9d02-38070ff2d7b9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:00.382183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c61b96-cdaa-48cc-9d02-38070ff2d7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:00.405305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 866/1736] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:00.407403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e55c5-dd13-475c-856c-3434fbac2de3 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:00.407890 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e55c5-dd13-475c-856c-3434fbac2de3 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:00.408329 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e55c5-dd13-475c-856c-3434fbac2de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:00.455367 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d1c61b96-cdaa-48cc-9d02-38070ff2d7b9 None None] MFA Rules not processed for user `c0ba164b85cd4d0899a5f6c7f5b2b6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:00.456635 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d1c61b96-cdaa-48cc-9d02-38070ff2d7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:00.479461 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-6a6e55c5-dd13-475c-856c-3434fbac2de3 None None] MFA Rules not processed for user `e4916206c2ff42669276e0a93d6483ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:00.481470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6a6e55c5-dd13-475c-856c-3434fbac2de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:00.629953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 871/1737] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:00.631550 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e7f9f-491d-499a-8618-ffb354ab7e81 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:00.632013 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e7f9f-491d-499a-8618-ffb354ab7e81 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:00.632432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e7f9f-491d-499a-8618-ffb354ab7e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:00.665031 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:00.680338 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 867/1738] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:00.680338 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e55c5-dd13-475c-856c-3434fbac2de3 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:00.681890 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a03e7f9f-491d-499a-8618-ffb354ab7e81 None None] MFA Rules not processed for user `5771e9d0c5c94306b3ec571916acdcaa`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:00.683296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a03e7f9f-491d-499a-8618-ffb354ab7e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:00.700308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 872/1739] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:00.703114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d207a-a0b6-4380-a502-8020c58f847c None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:00.703630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d207a-a0b6-4380-a502-8020c58f847c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:00.705329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d207a-a0b6-4380-a502-8020c58f847c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:00.751053 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-db2d207a-a0b6-4380-a502-8020c58f847c None None] MFA Rules not processed for user `e4916206c2ff42669276e0a93d6483ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:00.752391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-db2d207a-a0b6-4380-a502-8020c58f847c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:00.771922 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 873/1740] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:00.773127 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d207a-a0b6-4380-a502-8020c58f847c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:00.781718 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-db2d207a-a0b6-4380-a502-8020c58f847c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:00.860106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e55c5-dd13-475c-856c-3434fbac2de3 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:00.869486 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69838f70-f39f-4d7d-b89c-46d95a351ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:00.871239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69838f70-f39f-4d7d-b89c-46d95a351ffe admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:00.871239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69838f70-f39f-4d7d-b89c-46d95a351ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:00.871563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69838f70-f39f-4d7d-b89c-46d95a351ffe admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:00.873345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69838f70-f39f-4d7d-b89c-46d95a351ffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:00.874067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69838f70-f39f-4d7d-b89c-46d95a351ffe admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:00.946113 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d207a-a0b6-4380-a502-8020c58f847c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:00.955075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2e13e-d137-470a-bb75-dc8e3f7cf6ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:00.955925 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2e13e-d137-470a-bb75-dc8e3f7cf6ff service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:00.956300 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2e13e-d137-470a-bb75-dc8e3f7cf6ff service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:00.956863 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2e13e-d137-470a-bb75-dc8e3f7cf6ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:00.969242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2e13e-d137-470a-bb75-dc8e3f7cf6ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:00.971183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2e13e-d137-470a-bb75-dc8e3f7cf6ff service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:00.982791 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 874/1741] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:00.997683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9649be-733b-465c-b524-6e4d0d9d6354 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:01.001647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9649be-733b-465c-b524-6e4d0d9d6354 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:01.001647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9649be-733b-465c-b524-6e4d0d9d6354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:01.013657 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 868/1742] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:00 2022] POST /identity/v3/projects => generated 355 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:01.013657 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69838f70-f39f-4d7d-b89c-46d95a351ffe admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:01.019520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-69838f70-f39f-4d7d-b89c-46d95a351ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:01.062319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-8a9649be-733b-465c-b524-6e4d0d9d6354 None None] MFA Rules not processed for user `beb41eb66e8140a6999b0bddab3c4afd`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:01.063800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8a9649be-733b-465c-b524-6e4d0d9d6354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:01.165004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69838f70-f39f-4d7d-b89c-46d95a351ffe admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:01.175414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd57694-b249-4c34-892e-91584386ca52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:01.176208 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd57694-b249-4c34-892e-91584386ca52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:01.180359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd57694-b249-4c34-892e-91584386ca52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:01.180359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd57694-b249-4c34-892e-91584386ca52 admin admin] PATH_INFO: `/v3/projects/66bdf3e7b1ed4565a08bbcaea3bc1ec8` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:01.181109 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd57694-b249-4c34-892e-91584386ca52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66bdf3e7b1ed4565a08bbcaea3bc1ec8)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:01.185636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd57694-b249-4c34-892e-91584386ca52 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:01.194854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:01.211868 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 875/1743] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:01.213396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9649be-733b-465c-b524-6e4d0d9d6354 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:01.248507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9649be-733b-465c-b524-6e4d0d9d6354 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:01.264412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800a2eb-04f4-413a-bce9-49a9d0890dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:01.265566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800a2eb-04f4-413a-bce9-49a9d0890dba admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:01.266130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800a2eb-04f4-413a-bce9-49a9d0890dba admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:01.266735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800a2eb-04f4-413a-bce9-49a9d0890dba admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:01.267694 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800a2eb-04f4-413a-bce9-49a9d0890dba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:01.271252 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800a2eb-04f4-413a-bce9-49a9d0890dba admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:01.368308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 876/1744] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:01 2022] POST /identity/v3/users => generated 284 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:01.374284 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800a2eb-04f4-413a-bce9-49a9d0890dba admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:01.380386 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1fd57694-b249-4c34-892e-91584386ca52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66bdf3e7b1ed4565a08bbcaea3bc1ec8', 'request_id': 'req-1fd57694-b249-4c34-892e-91584386ca52'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:01.387088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800a2eb-04f4-413a-bce9-49a9d0890dba admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:01.395521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1fd57694-b249-4c34-892e-91584386ca52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66bdf3e7b1ed4565a08bbcaea3bc1ec8', 'request_id': 'req-1fd57694-b249-4c34-892e-91584386ca52'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:01.409542 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1fd57694-b249-4c34-892e-91584386ca52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66bdf3e7b1ed4565a08bbcaea3bc1ec8', 'request_id': 'req-1fd57694-b249-4c34-892e-91584386ca52'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:01.487716 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a8487-5050-46f7-9097-77f0e18fffa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:01.488428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a8487-5050-46f7-9097-77f0e18fffa6 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:01.488929 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a8487-5050-46f7-9097-77f0e18fffa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:01.489280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a8487-5050-46f7-9097-77f0e18fffa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:01.490025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a8487-5050-46f7-9097-77f0e18fffa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:01.491841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a8487-5050-46f7-9097-77f0e18fffa6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:01.512277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 877/1745] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:01.513786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cf122-4876-43ab-8a44-b6b9ed459756 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:01.515071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cf122-4876-43ab-8a44-b6b9ed459756 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:01.515071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cf122-4876-43ab-8a44-b6b9ed459756 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:01.516190 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 878/1746] 10.222.0.91 () {54 vars in 912 bytes} [Thu Sep 8 14:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:00:01.517879 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cf122-4876-43ab-8a44-b6b9ed459756 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:01.533417 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cf122-4876-43ab-8a44-b6b9ed459756 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:01.538698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dc0b5-635c-44ef-88f8-d4a5cc69ecc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:01.539530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dc0b5-635c-44ef-88f8-d4a5cc69ecc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:01.540078 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dc0b5-635c-44ef-88f8-d4a5cc69ecc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:01.540446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dc0b5-635c-44ef-88f8-d4a5cc69ecc3 admin admin] PATH_INFO: `/v3/projects/ac031016944c4da0bb8f09142714b653/users/86111abadd1942bba409ed39ec281aec/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:01.552305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1fd57694-b249-4c34-892e-91584386ca52 admin admin] The token cache is being invalidate because project 66bdf3e7b1ed4565a08bbcaea3bc1ec8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:01.554214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1fd57694-b249-4c34-892e-91584386ca52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:01.555747 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1fd57694-b249-4c34-892e-91584386ca52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:01.560482 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51dc0b5-635c-44ef-88f8-d4a5cc69ecc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac031016944c4da0bb8f09142714b653, user_id=86111abadd1942bba409ed39ec281aec, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:01.561611 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 869/1747] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:01 2022] DELETE /identity/v3/projects/66bdf3e7b1ed4565a08bbcaea3bc1ec8 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:01.564161 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51dc0b5-635c-44ef-88f8-d4a5cc69ecc3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:01.564457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed89f0-fc5c-42fd-9a80-c4867a0994f4 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:01.565124 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed89f0-fc5c-42fd-9a80-c4867a0994f4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:01.565814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed89f0-fc5c-42fd-9a80-c4867a0994f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:01.577855 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:01.622093 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8bed89f0-fc5c-42fd-9a80-c4867a0994f4 None None] MFA Rules not processed for user `cdbbaa64ad4844a0b2698dee82f95f23`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:01.623961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8bed89f0-fc5c-42fd-9a80-c4867a0994f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:01.655628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 879/1748] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:01 2022] PUT /identity/v3/projects/ac031016944c4da0bb8f09142714b653/users/86111abadd1942bba409ed39ec281aec/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:01.655628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cc8e2-1079-4f31-a938-ce732dce5cdb None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:01.655628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cc8e2-1079-4f31-a938-ce732dce5cdb None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:01.655628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cc8e2-1079-4f31-a938-ce732dce5cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:01.693872 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c74cc8e2-1079-4f31-a938-ce732dce5cdb None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:01.696313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c74cc8e2-1079-4f31-a938-ce732dce5cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:01.754735 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 870/1749] 10.222.0.91 () {58 vars in 1009 bytes} [Thu Sep 8 14:00:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:01.755952 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed89f0-fc5c-42fd-9a80-c4867a0994f4 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:01.761742 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8bed89f0-fc5c-42fd-9a80-c4867a0994f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:01.804897 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed89f0-fc5c-42fd-9a80-c4867a0994f4 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:01.849692 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 880/1750] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:01.850606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cc8e2-1079-4f31-a938-ce732dce5cdb None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:01.859888 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c74cc8e2-1079-4f31-a938-ce732dce5cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:01.930054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7a293-75bf-41a2-a21c-656c6e5160a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:01.930745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7a293-75bf-41a2-a21c-656c6e5160a1 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:01.931233 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7a293-75bf-41a2-a21c-656c6e5160a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:01.931603 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7a293-75bf-41a2-a21c-656c6e5160a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:01.932675 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7a293-75bf-41a2-a21c-656c6e5160a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:01.934237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7a293-75bf-41a2-a21c-656c6e5160a1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:01.946144 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 871/1751] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:01 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:01.947973 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662ce54-a826-4603-b0e5-26f55ebf92ed None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:01.948296 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662ce54-a826-4603-b0e5-26f55ebf92ed None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:01.948572 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662ce54-a826-4603-b0e5-26f55ebf92ed None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:01.950545 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 872/1752] 10.222.0.91 () {54 vars in 912 bytes} [Thu Sep 8 14:00:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:00:01.951722 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bdda1-539d-47e1-ba95-8dcf79d43ebb None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:01.951985 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bdda1-539d-47e1-ba95-8dcf79d43ebb None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:01.952253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bdda1-539d-47e1-ba95-8dcf79d43ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:01.959741 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cc8e2-1079-4f31-a938-ce732dce5cdb None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:01.964432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6083836-041c-4bf8-a5f0-9ae45236d60a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:01.965058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6083836-041c-4bf8-a5f0-9ae45236d60a service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:01.965338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6083836-041c-4bf8-a5f0-9ae45236d60a service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:01.965751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6083836-041c-4bf8-a5f0-9ae45236d60a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:01.973660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6083836-041c-4bf8-a5f0-9ae45236d60a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:01.975010 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-679bdda1-539d-47e1-ba95-8dcf79d43ebb None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:01.975255 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6083836-041c-4bf8-a5f0-9ae45236d60a service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:01.976416 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-679bdda1-539d-47e1-ba95-8dcf79d43ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:01.985142 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 881/1753] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:01.985943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6083836-041c-4bf8-a5f0-9ae45236d60a service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:01.991114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 873/1754] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:01.992761 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb86373-789b-42fb-bb2e-43c73dda7c0d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:01.993046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb86373-789b-42fb-bb2e-43c73dda7c0d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:01.993314 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb86373-789b-42fb-bb2e-43c73dda7c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:01.998782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6083836-041c-4bf8-a5f0-9ae45236d60a service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:02.011123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5e666-2875-4b92-aaa7-d3ed3f75ae8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:02.012700 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5e666-2875-4b92-aaa7-d3ed3f75ae8c admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:02.012700 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5e666-2875-4b92-aaa7-d3ed3f75ae8c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:02.012700 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5e666-2875-4b92-aaa7-d3ed3f75ae8c admin admin] PATH_INFO: `/v3/projects/ac031016944c4da0bb8f09142714b653/users/86111abadd1942bba409ed39ec281aec/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:02.014870 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5e666-2875-4b92-aaa7-d3ed3f75ae8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac031016944c4da0bb8f09142714b653, user_id=86111abadd1942bba409ed39ec281aec, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:02.017871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5e666-2875-4b92-aaa7-d3ed3f75ae8c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:02.024101 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-5cb86373-789b-42fb-bb2e-43c73dda7c0d None None] MFA Rules not processed for user `cdbbaa64ad4844a0b2698dee82f95f23`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:02.026010 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5cb86373-789b-42fb-bb2e-43c73dda7c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:02.046810 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 882/1755] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:01 2022] PUT /identity/v3/projects/ac031016944c4da0bb8f09142714b653/users/86111abadd1942bba409ed39ec281aec/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:02.046810 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5e666-2875-4b92-aaa7-d3ed3f75ae8c admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:02.136884 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 874/1756] 10.222.0.91 () {58 vars in 1009 bytes} [Thu Sep 8 14:00:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:02.138848 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc3188-69e6-4882-a92b-ec843a34f3a1 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:02.139531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc3188-69e6-4882-a92b-ec843a34f3a1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:02.140042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc3188-69e6-4882-a92b-ec843a34f3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:02.188001 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5e666-2875-4b92-aaa7-d3ed3f75ae8c admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:02.203678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0ee2e-c5b2-44bf-bdd4-9d040bc59e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:02.204089 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0ee2e-c5b2-44bf-bdd4-9d040bc59e96 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:02.204547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0ee2e-c5b2-44bf-bdd4-9d040bc59e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:02.204931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0ee2e-c5b2-44bf-bdd4-9d040bc59e96 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:02.207560 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0ee2e-c5b2-44bf-bdd4-9d040bc59e96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:00:02.207560 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0ee2e-c5b2-44bf-bdd4-9d040bc59e96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:02.208816 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0ee2e-c5b2-44bf-bdd4-9d040bc59e96 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:02.214646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-46bc3188-69e6-4882-a92b-ec843a34f3a1 None None] MFA Rules not processed for user `86111abadd1942bba409ed39ec281aec`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:02.216893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-46bc3188-69e6-4882-a92b-ec843a34f3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:02.244236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:02.248569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 883/1757] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:00:02.250996 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0ee2e-c5b2-44bf-bdd4-9d040bc59e96 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:02.258693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c9a0ee2e-c5b2-44bf-bdd4-9d040bc59e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:02.288477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0ee2e-c5b2-44bf-bdd4-9d040bc59e96 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:02.298876 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11558017-0cf4-4f37-a0ea-24ab8d12c89b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:02.299540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11558017-0cf4-4f37-a0ea-24ab8d12c89b service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:02.299815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11558017-0cf4-4f37-a0ea-24ab8d12c89b service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:02.300082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11558017-0cf4-4f37-a0ea-24ab8d12c89b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:02.322028 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11558017-0cf4-4f37-a0ea-24ab8d12c89b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:02.324193 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11558017-0cf4-4f37-a0ea-24ab8d12c89b service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:02.346305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 884/1758] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:02.347244 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11558017-0cf4-4f37-a0ea-24ab8d12c89b service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:02.376987 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11558017-0cf4-4f37-a0ea-24ab8d12c89b service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:02.388529 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4db164-4e1a-42f0-8149-4af0d35a1083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:02.389199 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4db164-4e1a-42f0-8149-4af0d35a1083 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:02.389524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4db164-4e1a-42f0-8149-4af0d35a1083 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:02.389794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4db164-4e1a-42f0-8149-4af0d35a1083 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:02.390509 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4db164-4e1a-42f0-8149-4af0d35a1083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:02.392830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4db164-4e1a-42f0-8149-4af0d35a1083 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:02.425810 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 875/1759] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:02.436442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc3188-69e6-4882-a92b-ec843a34f3a1 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:02.438396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-46bc3188-69e6-4882-a92b-ec843a34f3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:02.573151 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc3188-69e6-4882-a92b-ec843a34f3a1 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:02.580379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 885/1760] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:02 2022] POST /identity/v3/projects => generated 365 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:02.584173 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4db164-4e1a-42f0-8149-4af0d35a1083 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:02.587041 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c436b8d-42c2-498b-a741-59134ee7a92a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:02.587685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c436b8d-42c2-498b-a741-59134ee7a92a service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:02.587966 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c436b8d-42c2-498b-a741-59134ee7a92a service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:02.588237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c436b8d-42c2-498b-a741-59134ee7a92a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:02.640474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c436b8d-42c2-498b-a741-59134ee7a92a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:02.644851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c436b8d-42c2-498b-a741-59134ee7a92a service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:02.676719 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 876/1761] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:02.754960 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4db164-4e1a-42f0-8149-4af0d35a1083 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:02.769631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001498b5-3171-4865-85c7-ef088ad8e460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:02.770326 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001498b5-3171-4865-85c7-ef088ad8e460 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:02.770609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001498b5-3171-4865-85c7-ef088ad8e460 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:02.770973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001498b5-3171-4865-85c7-ef088ad8e460 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:02.771704 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001498b5-3171-4865-85c7-ef088ad8e460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:02.777247 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001498b5-3171-4865-85c7-ef088ad8e460 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:02.779174 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c436b8d-42c2-498b-a741-59134ee7a92a service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:02.780749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8c436b8d-42c2-498b-a741-59134ee7a92a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:03.001907 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c436b8d-42c2-498b-a741-59134ee7a92a service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:03.030450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 886/1762] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:02 2022] POST /identity/v3/users => generated 283 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:03.030450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001498b5-3171-4865-85c7-ef088ad8e460 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:03.034513 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-001498b5-3171-4865-85c7-ef088ad8e460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:03.120871 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ecfbc-127e-46f7-a6a0-02eab834d105 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:03.121665 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ecfbc-127e-46f7-a6a0-02eab834d105 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:03.122544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ecfbc-127e-46f7-a6a0-02eab834d105 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:03.122829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ecfbc-127e-46f7-a6a0-02eab834d105 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:03.124676 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8f7ecfbc-127e-46f7-a6a0-02eab834d105 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:03.188864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001498b5-3171-4865-85c7-ef088ad8e460 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:03.196905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fefbcbd-9975-49e2-aff4-b0b202d301bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9506139d5744fe69153e7e30a05b436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:03.197886 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefbcbd-9975-49e2-aff4-b0b202d301bf service placement] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:03.198975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefbcbd-9975-49e2-aff4-b0b202d301bf service placement] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:03.199895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefbcbd-9975-49e2-aff4-b0b202d301bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:03.203663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4fefbcbd-9975-49e2-aff4-b0b202d301bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:03.224471 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fefbcbd-9975-49e2-aff4-b0b202d301bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:03.227355 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fefbcbd-9975-49e2-aff4-b0b202d301bf service placement] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:03.245990 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 887/1763] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:03.247724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fefbcbd-9975-49e2-aff4-b0b202d301bf service placement] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:03.279780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fefbcbd-9975-49e2-aff4-b0b202d301bf service placement] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:03.286538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0140b37-b06e-4e2f-91c4-a545b94a77c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:03.287247 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0140b37-b06e-4e2f-91c4-a545b94a77c9 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:03.287528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0140b37-b06e-4e2f-91c4-a545b94a77c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:03.287895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0140b37-b06e-4e2f-91c4-a545b94a77c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:03.288107 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ecfbc-127e-46f7-a6a0-02eab834d105 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:03.288542 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0140b37-b06e-4e2f-91c4-a545b94a77c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:03.292537 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0140b37-b06e-4e2f-91c4-a545b94a77c9 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:03.300037 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ecfbc-127e-46f7-a6a0-02eab834d105 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:03.313243 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 888/1764] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:03.316505 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0140b37-b06e-4e2f-91c4-a545b94a77c9 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:03.325000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b0140b37-b06e-4e2f-91c4-a545b94a77c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:03.342814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 877/1765] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:03.344047 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ecfbc-127e-46f7-a6a0-02eab834d105 service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:03.366914 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0140b37-b06e-4e2f-91c4-a545b94a77c9 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:03.368811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ecfbc-127e-46f7-a6a0-02eab834d105 service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:03.386988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e2027-b696-491b-914e-be0dd62e47ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:03.387844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e2027-b696-491b-914e-be0dd62e47ca admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:03.388218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e2027-b696-491b-914e-be0dd62e47ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:03.389258 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e2027-b696-491b-914e-be0dd62e47ca admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:03.389549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e2027-b696-491b-914e-be0dd62e47ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:03.391836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3e2027-b696-491b-914e-be0dd62e47ca admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:03.408539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271846a-81df-45dd-ad7f-df3ebdf9bdf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:03.408729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271846a-81df-45dd-ad7f-df3ebdf9bdf1 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:03.408963 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271846a-81df-45dd-ad7f-df3ebdf9bdf1 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:03.409239 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271846a-81df-45dd-ad7f-df3ebdf9bdf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:03.418193 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8271846a-81df-45dd-ad7f-df3ebdf9bdf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:03.470818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 878/1766] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:03.488518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e2027-b696-491b-914e-be0dd62e47ca admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:03.524574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e2027-b696-491b-914e-be0dd62e47ca admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:03.558462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da79cb3-0bcf-488e-a073-4b91a775ff88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:03.558462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da79cb3-0bcf-488e-a073-4b91a775ff88 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:03.558462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da79cb3-0bcf-488e-a073-4b91a775ff88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:03.558462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da79cb3-0bcf-488e-a073-4b91a775ff88 admin admin] PATH_INFO: `/v3/projects/8cea32a8aa7a4b20a5f53e7643d08b20/users/226b44d97f254a5791e6636b7d04e4fd/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:03.602776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da79cb3-0bcf-488e-a073-4b91a775ff88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cea32a8aa7a4b20a5f53e7643d08b20, user_id=226b44d97f254a5791e6636b7d04e4fd, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:03.610792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da79cb3-0bcf-488e-a073-4b91a775ff88 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:03.841423 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271846a-81df-45dd-ad7f-df3ebdf9bdf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:03.841423 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271846a-81df-45dd-ad7f-df3ebdf9bdf1 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:03.841423 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 889/1767] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:03.841423 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271846a-81df-45dd-ad7f-df3ebdf9bdf1 service nova] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:04.124894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271846a-81df-45dd-ad7f-df3ebdf9bdf1 service nova] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:04.124894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ff293-64a3-4817-bf3c-257d87084ef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:04.124894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ff293-64a3-4817-bf3c-257d87084ef7 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:04.124894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ff293-64a3-4817-bf3c-257d87084ef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:04.126295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ff293-64a3-4817-bf3c-257d87084ef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:04.126295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-405ff293-64a3-4817-bf3c-257d87084ef7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:04.126295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ff293-64a3-4817-bf3c-257d87084ef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:04.126295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ff293-64a3-4817-bf3c-257d87084ef7 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:04.126295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 890/1768] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:03 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:04.149441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 879/1769] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:03 2022] PUT /identity/v3/projects/8cea32a8aa7a4b20a5f53e7643d08b20/users/226b44d97f254a5791e6636b7d04e4fd/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:04.151317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb2f33-4b8c-4f88-a4fe-d8be967bf02a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:04.151623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb2f33-4b8c-4f88-a4fe-d8be967bf02a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:04.152232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb2f33-4b8c-4f88-a4fe-d8be967bf02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:04.191914 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b6bb2f33-4b8c-4f88-a4fe-d8be967bf02a None None] MFA Rules not processed for user `226b44d97f254a5791e6636b7d04e4fd`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:04.191914 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b6bb2f33-4b8c-4f88-a4fe-d8be967bf02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:04.289985 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 891/1770] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:04.297321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da79cb3-0bcf-488e-a073-4b91a775ff88 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:04.529921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da79cb3-0bcf-488e-a073-4b91a775ff88 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:04.594850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841074fe-9189-4266-9c30-f426abe7c417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:04.594850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841074fe-9189-4266-9c30-f426abe7c417 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:04.594850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841074fe-9189-4266-9c30-f426abe7c417 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:04.594850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841074fe-9189-4266-9c30-f426abe7c417 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:04.595616 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841074fe-9189-4266-9c30-f426abe7c417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:04.599463 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841074fe-9189-4266-9c30-f426abe7c417 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:04.826238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 880/1771] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:04 2022] POST /identity/v3/projects => generated 365 bytes in 530 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:04.833378 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb2f33-4b8c-4f88-a4fe-d8be967bf02a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:04.874078 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb2f33-4b8c-4f88-a4fe-d8be967bf02a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:05.173853 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3d38e-e0fc-4178-9bb3-090417001420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:05.187879 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3d38e-e0fc-4178-9bb3-090417001420 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:05.190191 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3d38e-e0fc-4178-9bb3-090417001420 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:05.190453 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3d38e-e0fc-4178-9bb3-090417001420 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:05.193412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3d38e-e0fc-4178-9bb3-090417001420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:05.195428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3d38e-e0fc-4178-9bb3-090417001420 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:05.327998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a343af-0b68-44e3-90f3-771bc3df3cc8 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:05.328294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a343af-0b68-44e3-90f3-771bc3df3cc8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:05.328552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a343af-0b68-44e3-90f3-771bc3df3cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:05.355673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a1a343af-0b68-44e3-90f3-771bc3df3cc8 None None] MFA Rules not processed for user `e4916206c2ff42669276e0a93d6483ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:05.356946 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a1a343af-0b68-44e3-90f3-771bc3df3cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:05.365701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 892/1772] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:04 2022] POST /identity/v3/users => generated 289 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:05.370426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3d38e-e0fc-4178-9bb3-090417001420 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:05.398280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3d38e-e0fc-4178-9bb3-090417001420 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:05.410856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f6dff-592b-45f5-9c30-a23b27b4bad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:05.417538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f6dff-592b-45f5-9c30-a23b27b4bad6 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:05.417538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f6dff-592b-45f5-9c30-a23b27b4bad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:05.417538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f6dff-592b-45f5-9c30-a23b27b4bad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:05.417538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f6dff-592b-45f5-9c30-a23b27b4bad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:05.417538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f6dff-592b-45f5-9c30-a23b27b4bad6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:05.439305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:05.440411 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 893/1773] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:05.448739 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f6dff-592b-45f5-9c30-a23b27b4bad6 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:05.469507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f6dff-592b-45f5-9c30-a23b27b4bad6 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:05.494287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074eeab5-0889-44c5-b822-04c6bac2dfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:05.495047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074eeab5-0889-44c5-b822-04c6bac2dfa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:05.495330 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074eeab5-0889-44c5-b822-04c6bac2dfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:05.495778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074eeab5-0889-44c5-b822-04c6bac2dfa7 admin admin] PATH_INFO: `/v3/projects/86afe3ee752f481d90dfffb470872f1b/users/42a1d9567cf846e0bea32ec530cbd9e4/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:05.507774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 881/1774] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:05.536844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074eeab5-0889-44c5-b822-04c6bac2dfa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86afe3ee752f481d90dfffb470872f1b, user_id=42a1d9567cf846e0bea32ec530cbd9e4, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:05.549555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074eeab5-0889-44c5-b822-04c6bac2dfa7 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:05.701273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 894/1775] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:05 2022] PUT /identity/v3/projects/86afe3ee752f481d90dfffb470872f1b/users/42a1d9567cf846e0bea32ec530cbd9e4/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:05.708194 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a343af-0b68-44e3-90f3-771bc3df3cc8 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:05.733078 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a343af-0b68-44e3-90f3-771bc3df3cc8 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:05.944292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6621da3-5138-42b4-a26e-19fd532d5da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:05.944292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6621da3-5138-42b4-a26e-19fd532d5da1 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:05.944292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6621da3-5138-42b4-a26e-19fd532d5da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:05.949298 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6621da3-5138-42b4-a26e-19fd532d5da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:05.954832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6621da3-5138-42b4-a26e-19fd532d5da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:05.964545 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6621da3-5138-42b4-a26e-19fd532d5da1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:05.993911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 882/1776] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:06.000947 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074eeab5-0889-44c5-b822-04c6bac2dfa7 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:06.029234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074eeab5-0889-44c5-b822-04c6bac2dfa7 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:06.048538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f4954-9441-4071-9e78-b00c99b606fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:06.049473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f4954-9441-4071-9e78-b00c99b606fe admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:06.049933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f4954-9441-4071-9e78-b00c99b606fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:06.050530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f4954-9441-4071-9e78-b00c99b606fe admin admin] PATH_INFO: `/v3/projects/86afe3ee752f481d90dfffb470872f1b/users/42a1d9567cf846e0bea32ec530cbd9e4/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:06.054742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f4954-9441-4071-9e78-b00c99b606fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=86afe3ee752f481d90dfffb470872f1b, user_id=42a1d9567cf846e0bea32ec530cbd9e4, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:06.060088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f4954-9441-4071-9e78-b00c99b606fe admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:06.109885 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 895/1777] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:05 2022] PUT /identity/v3/projects/86afe3ee752f481d90dfffb470872f1b/users/42a1d9567cf846e0bea32ec530cbd9e4/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:06.125975 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3712369-dc44-4106-a68f-ca2cd3a8b696 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:06.127179 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3712369-dc44-4106-a68f-ca2cd3a8b696 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:06.127821 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3712369-dc44-4106-a68f-ca2cd3a8b696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:06.185494 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b3712369-dc44-4106-a68f-ca2cd3a8b696 None None] MFA Rules not processed for user `42a1d9567cf846e0bea32ec530cbd9e4`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:06.192678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b3712369-dc44-4106-a68f-ca2cd3a8b696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:06.404770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 883/1778] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:06.457731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f4954-9441-4071-9e78-b00c99b606fe admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:06.652463 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f4954-9441-4071-9e78-b00c99b606fe admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:06.680742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae8531-6d26-4d97-9258-6807241a824c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:06.681383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae8531-6d26-4d97-9258-6807241a824c service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:06.681666 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae8531-6d26-4d97-9258-6807241a824c service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:06.682121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae8531-6d26-4d97-9258-6807241a824c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:06.707631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae8531-6d26-4d97-9258-6807241a824c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:06.708537 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae8531-6d26-4d97-9258-6807241a824c service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:06.723171 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3712369-dc44-4106-a68f-ca2cd3a8b696 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:06.726488 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b3712369-dc44-4106-a68f-ca2cd3a8b696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:06.772863 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 896/1779] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:06 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:06.956326 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:06.971573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3712369-dc44-4106-a68f-ca2cd3a8b696 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:06.981203 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c50fb5-2c12-4bfb-96ab-67afdb3023db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:06.982620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c50fb5-2c12-4bfb-96ab-67afdb3023db admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:06.983225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c50fb5-2c12-4bfb-96ab-67afdb3023db admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:06.983983 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c50fb5-2c12-4bfb-96ab-67afdb3023db admin admin] PATH_INFO: `/v3/users/2c6492631396418283e4ff1a1ae6289b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:06.986379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c50fb5-2c12-4bfb-96ab-67afdb3023db admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c6492631396418283e4ff1a1ae6289b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:06.989181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c50fb5-2c12-4bfb-96ab-67afdb3023db admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:07.378465 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c5c50fb5-2c12-4bfb-96ab-67afdb3023db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c6492631396418283e4ff1a1ae6289b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:07.523418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c5c50fb5-2c12-4bfb-96ab-67afdb3023db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c6492631396418283e4ff1a1ae6289b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:07.555724 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c5c50fb5-2c12-4bfb-96ab-67afdb3023db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c6492631396418283e4ff1a1ae6289b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:07.579668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c5c50fb5-2c12-4bfb-96ab-67afdb3023db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c6492631396418283e4ff1a1ae6289b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:07.618815 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c5c50fb5-2c12-4bfb-96ab-67afdb3023db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c6492631396418283e4ff1a1ae6289b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:07.639681 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 884/1780] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:06 2022] DELETE /identity/v3/users/2c6492631396418283e4ff1a1ae6289b => generated 0 bytes in 917 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:07.644461 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838d1f1-a89d-4213-99e6-3b7a0a1958de None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:07.645350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838d1f1-a89d-4213-99e6-3b7a0a1958de None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:07.645815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838d1f1-a89d-4213-99e6-3b7a0a1958de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:07.702848 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c50fb5-2c12-4bfb-96ab-67afdb3023db admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:07.704450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c5c50fb5-2c12-4bfb-96ab-67afdb3023db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:07.728371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c838d1f1-a89d-4213-99e6-3b7a0a1958de None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:07.736800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c838d1f1-a89d-4213-99e6-3b7a0a1958de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:07.921624 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c50fb5-2c12-4bfb-96ab-67afdb3023db admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:07.927073 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41482c7d-2d98-4016-8ab3-afa7fc80bef0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:07.928205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41482c7d-2d98-4016-8ab3-afa7fc80bef0 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:07.929769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41482c7d-2d98-4016-8ab3-afa7fc80bef0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:07.930360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41482c7d-2d98-4016-8ab3-afa7fc80bef0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:07.933181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-41482c7d-2d98-4016-8ab3-afa7fc80bef0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:07.947192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 897/1781] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:07.961390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838d1f1-a89d-4213-99e6-3b7a0a1958de None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:07.963192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c838d1f1-a89d-4213-99e6-3b7a0a1958de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:08.049246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41482c7d-2d98-4016-8ab3-afa7fc80bef0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:08.051776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41482c7d-2d98-4016-8ab3-afa7fc80bef0 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:08.071861 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 885/1782] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:08.092748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838d1f1-a89d-4213-99e6-3b7a0a1958de None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:08.104693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0579114-c7fb-4e8c-bf48-50348f3f4d85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:08.105721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0579114-c7fb-4e8c-bf48-50348f3f4d85 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:08.105923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0579114-c7fb-4e8c-bf48-50348f3f4d85 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:08.106089 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0579114-c7fb-4e8c-bf48-50348f3f4d85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:08.122610 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0579114-c7fb-4e8c-bf48-50348f3f4d85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:08.124087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0579114-c7fb-4e8c-bf48-50348f3f4d85 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:08.147481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 898/1783] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:08.782515 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41482c7d-2d98-4016-8ab3-afa7fc80bef0 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:08.784841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-41482c7d-2d98-4016-8ab3-afa7fc80bef0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:08.815153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41482c7d-2d98-4016-8ab3-afa7fc80bef0 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:08.833574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3dead4-25c0-4200-bac1-5e0131313887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:08.834665 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3dead4-25c0-4200-bac1-5e0131313887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:08.835169 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3dead4-25c0-4200-bac1-5e0131313887 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:08.835988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3dead4-25c0-4200-bac1-5e0131313887 admin admin] PATH_INFO: `/v3/projects/adb7737a46da4ad6aa1dfc1c7974f346` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:08.842904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3dead4-25c0-4200-bac1-5e0131313887 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adb7737a46da4ad6aa1dfc1c7974f346)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:08.845467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3dead4-25c0-4200-bac1-5e0131313887 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:09.127060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7d3dead4-25c0-4200-bac1-5e0131313887 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adb7737a46da4ad6aa1dfc1c7974f346', 'request_id': 'req-7d3dead4-25c0-4200-bac1-5e0131313887'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:09.144488 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7d3dead4-25c0-4200-bac1-5e0131313887 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adb7737a46da4ad6aa1dfc1c7974f346', 'request_id': 'req-7d3dead4-25c0-4200-bac1-5e0131313887'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:09.153417 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7d3dead4-25c0-4200-bac1-5e0131313887 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adb7737a46da4ad6aa1dfc1c7974f346', 'request_id': 'req-7d3dead4-25c0-4200-bac1-5e0131313887'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:09.278128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7d3dead4-25c0-4200-bac1-5e0131313887 admin admin] The token cache is being invalidate because project adb7737a46da4ad6aa1dfc1c7974f346 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:09.278727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7d3dead4-25c0-4200-bac1-5e0131313887 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:09.279706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7d3dead4-25c0-4200-bac1-5e0131313887 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:09.281710 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 886/1784] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:08 2022] DELETE /identity/v3/projects/adb7737a46da4ad6aa1dfc1c7974f346 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:09.288511 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d255d-0c9f-409c-affe-27fd46bf4d0c None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:09.289831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d255d-0c9f-409c-affe-27fd46bf4d0c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:09.290338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d255d-0c9f-409c-affe-27fd46bf4d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:09.322598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-cf4d255d-0c9f-409c-affe-27fd46bf4d0c None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:09.324557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-cf4d255d-0c9f-409c-affe-27fd46bf4d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:09.392241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 899/1785] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:09.400811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823af0a-d787-484a-bc9e-78b25940ab23 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:09.401122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823af0a-d787-484a-bc9e-78b25940ab23 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:09.401589 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823af0a-d787-484a-bc9e-78b25940ab23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:09.431112 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f823af0a-d787-484a-bc9e-78b25940ab23 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:09.433419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f823af0a-d787-484a-bc9e-78b25940ab23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:09.444986 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 887/1786] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:09.450456 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d255d-0c9f-409c-affe-27fd46bf4d0c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:09.470821 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d255d-0c9f-409c-affe-27fd46bf4d0c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:09.471008 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44c6db-d26d-4c5e-a96a-f22393c60f81 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:09.471255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44c6db-d26d-4c5e-a96a-f22393c60f81 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:09.472070 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44c6db-d26d-4c5e-a96a-f22393c60f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:09.481118 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a65951-cdf0-4059-a963-0646f8ecb992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:09.496123 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a65951-cdf0-4059-a963-0646f8ecb992 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:09.497027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a65951-cdf0-4059-a963-0646f8ecb992 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:09.497539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a65951-cdf0-4059-a963-0646f8ecb992 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:09.498555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a65951-cdf0-4059-a963-0646f8ecb992 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:00:09.499144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a65951-cdf0-4059-a963-0646f8ecb992 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:09.502255 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a65951-cdf0-4059-a963-0646f8ecb992 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:09.521006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 900/1787] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:00:09.525762 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a65951-cdf0-4059-a963-0646f8ecb992 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:09.538466 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ec44c6db-d26d-4c5e-a96a-f22393c60f81 None None] MFA Rules not processed for user `beb41eb66e8140a6999b0bddab3c4afd`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:09.539575 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a65951-cdf0-4059-a963-0646f8ecb992 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:09.539862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ec44c6db-d26d-4c5e-a96a-f22393c60f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:09.546163 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffd3c4-b24e-4839-a43c-8c65048b97ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:09.553693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffd3c4-b24e-4839-a43c-8c65048b97ab admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:09.553693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffd3c4-b24e-4839-a43c-8c65048b97ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:09.553693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffd3c4-b24e-4839-a43c-8c65048b97ab admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:09.553693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffd3c4-b24e-4839-a43c-8c65048b97ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:09.553693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffd3c4-b24e-4839-a43c-8c65048b97ab admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:09.614649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 888/1788] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:09.615930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44c6db-d26d-4c5e-a96a-f22393c60f81 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:09.617472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ec44c6db-d26d-4c5e-a96a-f22393c60f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:09.640032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44c6db-d26d-4c5e-a96a-f22393c60f81 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:09.648078 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0dd209-6c3e-4b37-a766-cb684ae7daa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:09.649542 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dd209-6c3e-4b37-a766-cb684ae7daa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:09.650714 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dd209-6c3e-4b37-a766-cb684ae7daa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:09.651450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dd209-6c3e-4b37-a766-cb684ae7daa6 admin admin] PATH_INFO: `/v3/users/9913db0bd00f4560ae6c595b35bc92ec` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:09.655311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0dd209-6c3e-4b37-a766-cb684ae7daa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9913db0bd00f4560ae6c595b35bc92ec)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:09.658125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0dd209-6c3e-4b37-a766-cb684ae7daa6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:09.662788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 901/1789] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:09 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:00:09.668100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffd3c4-b24e-4839-a43c-8c65048b97ab admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:09.793431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffd3c4-b24e-4839-a43c-8c65048b97ab admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:09.799627 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce094fc9-a036-46b1-bd81-a5f903d59c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:09.800282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce094fc9-a036-46b1-bd81-a5f903d59c58 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:09.800547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce094fc9-a036-46b1-bd81-a5f903d59c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:09.800820 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce094fc9-a036-46b1-bd81-a5f903d59c58 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:09.803186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce094fc9-a036-46b1-bd81-a5f903d59c58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:09.805854 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce094fc9-a036-46b1-bd81-a5f903d59c58 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:09.948697 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fe0dd209-6c3e-4b37-a766-cb684ae7daa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9913db0bd00f4560ae6c595b35bc92ec'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:10.021450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 902/1790] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:09 2022] POST /identity/v3/users => generated 287 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:10.026344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce094fc9-a036-46b1-bd81-a5f903d59c58 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:10.058370 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce094fc9-a036-46b1-bd81-a5f903d59c58 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:10.087862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196bf5d-4910-4e61-825d-07e400ef9f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:10.088871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196bf5d-4910-4e61-825d-07e400ef9f99 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:10.089335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196bf5d-4910-4e61-825d-07e400ef9f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:10.089983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196bf5d-4910-4e61-825d-07e400ef9f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:10.090430 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196bf5d-4910-4e61-825d-07e400ef9f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:10.095748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196bf5d-4910-4e61-825d-07e400ef9f99 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:10.114165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fe0dd209-6c3e-4b37-a766-cb684ae7daa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9913db0bd00f4560ae6c595b35bc92ec'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:10.115109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 903/1791] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:10.116220 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196bf5d-4910-4e61-825d-07e400ef9f99 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:10.164354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fe0dd209-6c3e-4b37-a766-cb684ae7daa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9913db0bd00f4560ae6c595b35bc92ec'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:10.168671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fe0dd209-6c3e-4b37-a766-cb684ae7daa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9913db0bd00f4560ae6c595b35bc92ec'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:10.177179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196bf5d-4910-4e61-825d-07e400ef9f99 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:10.179431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a45f1-9df8-4fed-84ad-f4b374c449f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:10.180459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a45f1-9df8-4fed-84ad-f4b374c449f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:10.181272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a45f1-9df8-4fed-84ad-f4b374c449f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:10.181779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a45f1-9df8-4fed-84ad-f4b374c449f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:10.183549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a45f1-9df8-4fed-84ad-f4b374c449f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:10.188328 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a45f1-9df8-4fed-84ad-f4b374c449f6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:10.211771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fe0dd209-6c3e-4b37-a766-cb684ae7daa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9913db0bd00f4560ae6c595b35bc92ec'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:10.234527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 889/1792] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:09 2022] DELETE /identity/v3/users/9913db0bd00f4560ae6c595b35bc92ec => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:10.241617 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73f1ad-c74f-4144-8dcc-63c1d471988b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:10.242176 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73f1ad-c74f-4144-8dcc-63c1d471988b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:10.242680 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73f1ad-c74f-4144-8dcc-63c1d471988b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:10.287153 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 904/1793] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:10.294302 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a45f1-9df8-4fed-84ad-f4b374c449f6 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:10.334029 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-041a45f1-9df8-4fed-84ad-f4b374c449f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:10.344553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ff73f1ad-c74f-4144-8dcc-63c1d471988b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:10.346146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ff73f1ad-c74f-4144-8dcc-63c1d471988b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:10.362402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a45f1-9df8-4fed-84ad-f4b374c449f6 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:10.474558 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:10.639637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883f802-cbd2-4d6d-89dc-14ad1753adb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:10.640338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883f802-cbd2-4d6d-89dc-14ad1753adb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:10.640609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883f802-cbd2-4d6d-89dc-14ad1753adb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:10.640892 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883f802-cbd2-4d6d-89dc-14ad1753adb6 admin admin] PATH_INFO: `/v3/projects/5d20f4ba94804b3a9de1dfa49e34126c/users/9ea4e41ed5b84be0be8e226b06953dd3/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:10.660518 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883f802-cbd2-4d6d-89dc-14ad1753adb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d20f4ba94804b3a9de1dfa49e34126c, user_id=9ea4e41ed5b84be0be8e226b06953dd3, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:10.663523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883f802-cbd2-4d6d-89dc-14ad1753adb6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:10.664980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 890/1794] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:10.688643 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73f1ad-c74f-4144-8dcc-63c1d471988b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:10.695927 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ff73f1ad-c74f-4144-8dcc-63c1d471988b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:10.773257 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 905/1795] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:10 2022] PUT /identity/v3/projects/5d20f4ba94804b3a9de1dfa49e34126c/users/9ea4e41ed5b84be0be8e226b06953dd3/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:10.794144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320cb5b-0d90-4685-bf1b-f83b045a3549 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:10.799432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320cb5b-0d90-4685-bf1b-f83b045a3549 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:10.799432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320cb5b-0d90-4685-bf1b-f83b045a3549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:10.868855 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-6320cb5b-0d90-4685-bf1b-f83b045a3549 None None] MFA Rules not processed for user `9ea4e41ed5b84be0be8e226b06953dd3`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:10.873037 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73f1ad-c74f-4144-8dcc-63c1d471988b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:10.874461 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6320cb5b-0d90-4685-bf1b-f83b045a3549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:11.037540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 906/1796] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:11.042407 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb6b58-9d58-4527-8465-ce15387dbe3f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:11.046406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f7b42-723b-4df0-8e8b-798a603424c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:11.046781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb6b58-9d58-4527-8465-ce15387dbe3f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:11.047424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb6b58-9d58-4527-8465-ce15387dbe3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:11.050502 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f7b42-723b-4df0-8e8b-798a603424c6 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:11.051712 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f7b42-723b-4df0-8e8b-798a603424c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:11.054240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f7b42-723b-4df0-8e8b-798a603424c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:11.212488 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f7b42-723b-4df0-8e8b-798a603424c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:11.216725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f7b42-723b-4df0-8e8b-798a603424c6 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:11.271234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 891/1797] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:00:11.311885 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-7feb6b58-9d58-4527-8465-ce15387dbe3f None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:11.318706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7feb6b58-9d58-4527-8465-ce15387dbe3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:11.559493 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 907/1798] 10.222.0.91 () {58 vars in 1009 bytes} [Thu Sep 8 14:00:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 520 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:11.657201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f7b42-723b-4df0-8e8b-798a603424c6 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:11.658954 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a03f7b42-723b-4df0-8e8b-798a603424c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:11.793408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:12.016903 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f7b42-723b-4df0-8e8b-798a603424c6 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:12.036758 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49068df-3808-48a5-a8e2-7e2f5341e4e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:12.037543 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49068df-3808-48a5-a8e2-7e2f5341e4e2 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:12.037962 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49068df-3808-48a5-a8e2-7e2f5341e4e2 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:12.038307 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49068df-3808-48a5-a8e2-7e2f5341e4e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:12.051068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49068df-3808-48a5-a8e2-7e2f5341e4e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:12.053133 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49068df-3808-48a5-a8e2-7e2f5341e4e2 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:12.069954 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 892/1799] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:00:12.180017 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb6b58-9d58-4527-8465-ce15387dbe3f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:12.181589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7feb6b58-9d58-4527-8465-ce15387dbe3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:12.442449 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb6b58-9d58-4527-8465-ce15387dbe3f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:12.457337 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d709238-47b4-4b76-8285-db3f75480d4b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:12.458655 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d709238-47b4-4b76-8285-db3f75480d4b service swift] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:12.459272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d709238-47b4-4b76-8285-db3f75480d4b service swift] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:12.459910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d709238-47b4-4b76-8285-db3f75480d4b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:12.486778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d709238-47b4-4b76-8285-db3f75480d4b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:12.501489 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d709238-47b4-4b76-8285-db3f75480d4b service swift] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:12.529390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 908/1800] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 14:00:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:00:12.752023 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49068df-3808-48a5-a8e2-7e2f5341e4e2 service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:12.777007 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49068df-3808-48a5-a8e2-7e2f5341e4e2 service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:12.790965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff2f07-58d8-41de-b9b7-88427ce25c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:12.792169 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff2f07-58d8-41de-b9b7-88427ce25c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:12.792449 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff2f07-58d8-41de-b9b7-88427ce25c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:12.792717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff2f07-58d8-41de-b9b7-88427ce25c2e admin admin] PATH_INFO: `/v3/users/9ea4e41ed5b84be0be8e226b06953dd3` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:12.795111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff2f07-58d8-41de-b9b7-88427ce25c2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ea4e41ed5b84be0be8e226b06953dd3)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:12.800742 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff2f07-58d8-41de-b9b7-88427ce25c2e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:13.325195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b7ff2f07-58d8-41de-b9b7-88427ce25c2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ea4e41ed5b84be0be8e226b06953dd3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:13.593381 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b7ff2f07-58d8-41de-b9b7-88427ce25c2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ea4e41ed5b84be0be8e226b06953dd3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:13.662374 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b7ff2f07-58d8-41de-b9b7-88427ce25c2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ea4e41ed5b84be0be8e226b06953dd3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:13.663780 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b7ff2f07-58d8-41de-b9b7-88427ce25c2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ea4e41ed5b84be0be8e226b06953dd3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:13.677837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b7ff2f07-58d8-41de-b9b7-88427ce25c2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ea4e41ed5b84be0be8e226b06953dd3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:13.695188 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 893/1801] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:12 2022] DELETE /identity/v3/users/9ea4e41ed5b84be0be8e226b06953dd3 => generated 0 bytes in 940 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:13.696930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee4f35-6fd9-488e-be06-52c29db2c6a3 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:13.697542 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee4f35-6fd9-488e-be06-52c29db2c6a3 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:13.697542 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee4f35-6fd9-488e-be06-52c29db2c6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:13.760391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1aee4f35-6fd9-488e-be06-52c29db2c6a3 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:13.765378 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1aee4f35-6fd9-488e-be06-52c29db2c6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:13.795892 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff2f07-58d8-41de-b9b7-88427ce25c2e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:13.799549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b7ff2f07-58d8-41de-b9b7-88427ce25c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:13.971622 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff2f07-58d8-41de-b9b7-88427ce25c2e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:13.980708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef7306-ed96-49ac-86c1-8a17a935ba52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:13.981474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef7306-ed96-49ac-86c1-8a17a935ba52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:13.981757 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef7306-ed96-49ac-86c1-8a17a935ba52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:13.982082 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef7306-ed96-49ac-86c1-8a17a935ba52 admin admin] PATH_INFO: `/v3/users/226b44d97f254a5791e6636b7d04e4fd` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:13.983791 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef7306-ed96-49ac-86c1-8a17a935ba52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=226b44d97f254a5791e6636b7d04e4fd)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:13.986838 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef7306-ed96-49ac-86c1-8a17a935ba52 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:13.989736 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 909/1802] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:14.050280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee4f35-6fd9-488e-be06-52c29db2c6a3 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:14.050280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1aee4f35-6fd9-488e-be06-52c29db2c6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:14.335117 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee4f35-6fd9-488e-be06-52c29db2c6a3 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:14.354234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98daf4-1c24-4843-9e24-d6074fc02a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:14.354877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98daf4-1c24-4843-9e24-d6074fc02a19 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:14.355211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98daf4-1c24-4843-9e24-d6074fc02a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:14.355465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98daf4-1c24-4843-9e24-d6074fc02a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:14.394134 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98daf4-1c24-4843-9e24-d6074fc02a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:14.400236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98daf4-1c24-4843-9e24-d6074fc02a19 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:14.429365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 910/1803] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:14.430412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98daf4-1c24-4843-9e24-d6074fc02a19 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:14.432003 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8b98daf4-1c24-4843-9e24-d6074fc02a19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:14.433025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ebef7306-ed96-49ac-86c1-8a17a935ba52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '226b44d97f254a5791e6636b7d04e4fd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:14.508657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98daf4-1c24-4843-9e24-d6074fc02a19 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:14.529001 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398300e-da50-419e-9ab9-40349cf471b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:14.529787 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398300e-da50-419e-9ab9-40349cf471b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:14.530138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398300e-da50-419e-9ab9-40349cf471b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:14.530304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398300e-da50-419e-9ab9-40349cf471b3 admin admin] PATH_INFO: `/v3/projects/f01d60ca831b4b2ab0b1fdea2cee9696` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:14.531255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ebef7306-ed96-49ac-86c1-8a17a935ba52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '226b44d97f254a5791e6636b7d04e4fd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:14.535808 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398300e-da50-419e-9ab9-40349cf471b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f01d60ca831b4b2ab0b1fdea2cee9696)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:14.545967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398300e-da50-419e-9ab9-40349cf471b3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:14.614987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ebef7306-ed96-49ac-86c1-8a17a935ba52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '226b44d97f254a5791e6636b7d04e4fd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:14.625082 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ebef7306-ed96-49ac-86c1-8a17a935ba52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '226b44d97f254a5791e6636b7d04e4fd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:14.663502 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ebef7306-ed96-49ac-86c1-8a17a935ba52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '226b44d97f254a5791e6636b7d04e4fd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:14.683953 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 894/1804] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:13 2022] DELETE /identity/v3/users/226b44d97f254a5791e6636b7d04e4fd => generated 0 bytes in 895 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:14.690815 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8804f8d-667e-4961-9716-0a0376f3189a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:14.691342 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8804f8d-667e-4961-9716-0a0376f3189a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:14.693228 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8804f8d-667e-4961-9716-0a0376f3189a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:14.744052 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d8804f8d-667e-4961-9716-0a0376f3189a None None] MFA Rules not processed for user `cdbbaa64ad4844a0b2698dee82f95f23`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:14.745771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d8804f8d-667e-4961-9716-0a0376f3189a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:14.799695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5398300e-da50-419e-9ab9-40349cf471b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f01d60ca831b4b2ab0b1fdea2cee9696', 'request_id': 'req-5398300e-da50-419e-9ab9-40349cf471b3'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:14.889066 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5398300e-da50-419e-9ab9-40349cf471b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f01d60ca831b4b2ab0b1fdea2cee9696', 'request_id': 'req-5398300e-da50-419e-9ab9-40349cf471b3'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:14.903987 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5398300e-da50-419e-9ab9-40349cf471b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f01d60ca831b4b2ab0b1fdea2cee9696', 'request_id': 'req-5398300e-da50-419e-9ab9-40349cf471b3'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:14.936532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5398300e-da50-419e-9ab9-40349cf471b3 admin admin] The token cache is being invalidate because project f01d60ca831b4b2ab0b1fdea2cee9696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:14.939585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:14.940109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5398300e-da50-419e-9ab9-40349cf471b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:14.940109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5398300e-da50-419e-9ab9-40349cf471b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:14.944648 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 911/1805] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:14 2022] DELETE /identity/v3/projects/f01d60ca831b4b2ab0b1fdea2cee9696 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:14.949408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c8b27-9d48-44f1-b8a3-ba51bbfd9b42 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:14.950672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c8b27-9d48-44f1-b8a3-ba51bbfd9b42 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:14.951877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c8b27-9d48-44f1-b8a3-ba51bbfd9b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:14.975779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 895/1806] 10.222.0.105 () {58 vars in 1016 bytes} [Thu Sep 8 14:00:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:14.983691 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8804f8d-667e-4961-9716-0a0376f3189a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:14.991466 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d8804f8d-667e-4961-9716-0a0376f3189a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:15.026844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ce7c8b27-9d48-44f1-b8a3-ba51bbfd9b42 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:15.034620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ce7c8b27-9d48-44f1-b8a3-ba51bbfd9b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:15.216190 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 912/1807] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:15.217870 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c8b27-9d48-44f1-b8a3-ba51bbfd9b42 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:15.226289 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ce7c8b27-9d48-44f1-b8a3-ba51bbfd9b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:15.275743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c8b27-9d48-44f1-b8a3-ba51bbfd9b42 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:15.284678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8804f8d-667e-4961-9716-0a0376f3189a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:15.294712 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01087170-c012-4c92-8bda-9ddd10c6d2f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:15.295630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01087170-c012-4c92-8bda-9ddd10c6d2f1 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:15.296057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01087170-c012-4c92-8bda-9ddd10c6d2f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:15.300425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01087170-c012-4c92-8bda-9ddd10c6d2f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:15.302504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e945bab-a536-49fd-9775-6dfa6c68eeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:15.303473 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e945bab-a536-49fd-9775-6dfa6c68eeff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:15.303970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e945bab-a536-49fd-9775-6dfa6c68eeff admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:15.304459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e945bab-a536-49fd-9775-6dfa6c68eeff admin admin] PATH_INFO: `/v3/users/d766ddfd9cb64363b667ed79096490d2` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:15.306332 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e945bab-a536-49fd-9775-6dfa6c68eeff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d766ddfd9cb64363b667ed79096490d2)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:15.309824 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e945bab-a536-49fd-9775-6dfa6c68eeff admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:15.332396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01087170-c012-4c92-8bda-9ddd10c6d2f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:15.332396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01087170-c012-4c92-8bda-9ddd10c6d2f1 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:15.353488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 913/1808] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:15.354878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01087170-c012-4c92-8bda-9ddd10c6d2f1 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:15.383180 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01087170-c012-4c92-8bda-9ddd10c6d2f1 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:15.400530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814aa8e-1453-440c-99be-f643fce23f82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:15.401267 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814aa8e-1453-440c-99be-f643fce23f82 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:15.401707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814aa8e-1453-440c-99be-f643fce23f82 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:15.402049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814aa8e-1453-440c-99be-f643fce23f82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:15.413874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814aa8e-1453-440c-99be-f643fce23f82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:15.415656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814aa8e-1453-440c-99be-f643fce23f82 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:15.429018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 914/1809] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:15.691211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1e945bab-a536-49fd-9775-6dfa6c68eeff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd766ddfd9cb64363b667ed79096490d2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:15.783632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1e945bab-a536-49fd-9775-6dfa6c68eeff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd766ddfd9cb64363b667ed79096490d2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:15.825722 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1e945bab-a536-49fd-9775-6dfa6c68eeff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd766ddfd9cb64363b667ed79096490d2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:15.825722 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1e945bab-a536-49fd-9775-6dfa6c68eeff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd766ddfd9cb64363b667ed79096490d2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:15.836314 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1e945bab-a536-49fd-9775-6dfa6c68eeff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd766ddfd9cb64363b667ed79096490d2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:15.875097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 896/1810] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:14 2022] DELETE /identity/v3/users/d766ddfd9cb64363b667ed79096490d2 => generated 0 bytes in 888 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:16.272219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:16.538718 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814aa8e-1453-440c-99be-f643fce23f82 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:16.541900 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6814aa8e-1453-440c-99be-f643fce23f82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:16.599499 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814aa8e-1453-440c-99be-f643fce23f82 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:16.705586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6054ae5-2a9d-43cd-8e9b-50349c019f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:16.707246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6054ae5-2a9d-43cd-8e9b-50349c019f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:16.709331 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6054ae5-2a9d-43cd-8e9b-50349c019f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:16.709906 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6054ae5-2a9d-43cd-8e9b-50349c019f88 admin admin] PATH_INFO: `/v3/projects/5d20f4ba94804b3a9de1dfa49e34126c` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:16.712244 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6054ae5-2a9d-43cd-8e9b-50349c019f88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d20f4ba94804b3a9de1dfa49e34126c)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:16.718290 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6054ae5-2a9d-43cd-8e9b-50349c019f88 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:16.998258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d6054ae5-2a9d-43cd-8e9b-50349c019f88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d20f4ba94804b3a9de1dfa49e34126c', 'request_id': 'req-d6054ae5-2a9d-43cd-8e9b-50349c019f88'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:17.175837 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d6054ae5-2a9d-43cd-8e9b-50349c019f88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d20f4ba94804b3a9de1dfa49e34126c', 'request_id': 'req-d6054ae5-2a9d-43cd-8e9b-50349c019f88'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:17.206394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d6054ae5-2a9d-43cd-8e9b-50349c019f88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d20f4ba94804b3a9de1dfa49e34126c', 'request_id': 'req-d6054ae5-2a9d-43cd-8e9b-50349c019f88'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:17.224777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d6054ae5-2a9d-43cd-8e9b-50349c019f88 admin admin] The token cache is being invalidate because project 5d20f4ba94804b3a9de1dfa49e34126c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:17.224777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d6054ae5-2a9d-43cd-8e9b-50349c019f88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:17.227317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d6054ae5-2a9d-43cd-8e9b-50349c019f88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:17.232965 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 915/1811] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:16 2022] DELETE /identity/v3/projects/5d20f4ba94804b3a9de1dfa49e34126c => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:17.238831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67901233-d819-4452-9f41-acb9f496113b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:17.239937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67901233-d819-4452-9f41-acb9f496113b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:17.240802 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67901233-d819-4452-9f41-acb9f496113b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:17.305117 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-67901233-d819-4452-9f41-acb9f496113b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:17.307572 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-67901233-d819-4452-9f41-acb9f496113b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:17.441225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 897/1812] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:17.443418 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bf8fa-0027-4367-bc9d-0ec99fe0fa5f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:17.443946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bf8fa-0027-4367-bc9d-0ec99fe0fa5f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:17.444099 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bf8fa-0027-4367-bc9d-0ec99fe0fa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:17.520474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-9b1bf8fa-0027-4367-bc9d-0ec99fe0fa5f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:17.522841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9b1bf8fa-0027-4367-bc9d-0ec99fe0fa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:17.536746 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 916/1813] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:17.541123 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67901233-d819-4452-9f41-acb9f496113b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:17.554061 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67901233-d819-4452-9f41-acb9f496113b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:17.561186 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cab7e9-33eb-4e4f-8ad9-2faf895656e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:17.562020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cab7e9-33eb-4e4f-8ad9-2faf895656e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:17.562551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cab7e9-33eb-4e4f-8ad9-2faf895656e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:17.562932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cab7e9-33eb-4e4f-8ad9-2faf895656e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:17.563731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cab7e9-33eb-4e4f-8ad9-2faf895656e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:00:17.564119 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cab7e9-33eb-4e4f-8ad9-2faf895656e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:17.565845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cab7e9-33eb-4e4f-8ad9-2faf895656e0 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:17.587433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 898/1814] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:00:17.593174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bf8fa-0027-4367-bc9d-0ec99fe0fa5f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:17.611614 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bf8fa-0027-4367-bc9d-0ec99fe0fa5f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:17.655062 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c87a6-db49-4215-9102-13e752beb8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:17.656216 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c87a6-db49-4215-9102-13e752beb8db admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:17.660326 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c87a6-db49-4215-9102-13e752beb8db admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:17.662022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c87a6-db49-4215-9102-13e752beb8db admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:17.666022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330c87a6-db49-4215-9102-13e752beb8db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:17.668789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330c87a6-db49-4215-9102-13e752beb8db admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:17.836317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 917/1815] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:17 2022] POST /identity/v3/projects => generated 377 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:17.837414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cab7e9-33eb-4e4f-8ad9-2faf895656e0 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:17.859774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cab7e9-33eb-4e4f-8ad9-2faf895656e0 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:18.070844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc26f6c-3da3-40b3-a1ff-7554a1784d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:18.071787 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc26f6c-3da3-40b3-a1ff-7554a1784d03 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:18.072391 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc26f6c-3da3-40b3-a1ff-7554a1784d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:18.073587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc26f6c-3da3-40b3-a1ff-7554a1784d03 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:18.074447 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc26f6c-3da3-40b3-a1ff-7554a1784d03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:18.076648 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc26f6c-3da3-40b3-a1ff-7554a1784d03 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:18.247075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c87a6-db49-4215-9102-13e752beb8db admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:18.249195 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-330c87a6-db49-4215-9102-13e752beb8db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:18.279341 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c87a6-db49-4215-9102-13e752beb8db admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:18.284957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80b22c-8a61-480c-adea-59dcaaf88316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:18.285948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80b22c-8a61-480c-adea-59dcaaf88316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:18.286233 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80b22c-8a61-480c-adea-59dcaaf88316 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:18.286499 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80b22c-8a61-480c-adea-59dcaaf88316 admin admin] PATH_INFO: `/v3/projects/8cea32a8aa7a4b20a5f53e7643d08b20` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:18.288129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80b22c-8a61-480c-adea-59dcaaf88316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cea32a8aa7a4b20a5f53e7643d08b20)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:18.290341 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80b22c-8a61-480c-adea-59dcaaf88316 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:18.498365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 899/1816] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:17 2022] POST /identity/v3/users => generated 289 bytes in 662 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:18.501317 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc26f6c-3da3-40b3-a1ff-7554a1784d03 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:18.504600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-edc26f6c-3da3-40b3-a1ff-7554a1784d03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:18.528459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc26f6c-3da3-40b3-a1ff-7554a1784d03 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:18.567251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abee0eb-099e-4367-b2c1-5040c7f9518e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:18.568136 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abee0eb-099e-4367-b2c1-5040c7f9518e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:18.568457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abee0eb-099e-4367-b2c1-5040c7f9518e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:18.568844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abee0eb-099e-4367-b2c1-5040c7f9518e admin admin] PATH_INFO: `/v3/projects/fe1541151785413590856ae0585ff8e1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:18.573223 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abee0eb-099e-4367-b2c1-5040c7f9518e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe1541151785413590856ae0585ff8e1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:18.578065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abee0eb-099e-4367-b2c1-5040c7f9518e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:19.093348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6a80b22c-8a61-480c-adea-59dcaaf88316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cea32a8aa7a4b20a5f53e7643d08b20', 'request_id': 'req-6a80b22c-8a61-480c-adea-59dcaaf88316'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:19.156549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7abee0eb-099e-4367-b2c1-5040c7f9518e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe1541151785413590856ae0585ff8e1', 'request_id': 'req-7abee0eb-099e-4367-b2c1-5040c7f9518e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:19.176040 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7abee0eb-099e-4367-b2c1-5040c7f9518e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe1541151785413590856ae0585ff8e1', 'request_id': 'req-7abee0eb-099e-4367-b2c1-5040c7f9518e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:19.195455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7abee0eb-099e-4367-b2c1-5040c7f9518e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe1541151785413590856ae0585ff8e1', 'request_id': 'req-7abee0eb-099e-4367-b2c1-5040c7f9518e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:19.341216 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6a80b22c-8a61-480c-adea-59dcaaf88316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cea32a8aa7a4b20a5f53e7643d08b20', 'request_id': 'req-6a80b22c-8a61-480c-adea-59dcaaf88316'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:19.368715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6a80b22c-8a61-480c-adea-59dcaaf88316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cea32a8aa7a4b20a5f53e7643d08b20', 'request_id': 'req-6a80b22c-8a61-480c-adea-59dcaaf88316'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:19.415159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6a80b22c-8a61-480c-adea-59dcaaf88316 admin admin] The token cache is being invalidate because project 8cea32a8aa7a4b20a5f53e7643d08b20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:19.415635 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6a80b22c-8a61-480c-adea-59dcaaf88316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:19.419043 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6a80b22c-8a61-480c-adea-59dcaaf88316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:19.425985 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 918/1817] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:18 2022] DELETE /identity/v3/projects/8cea32a8aa7a4b20a5f53e7643d08b20 => generated 0 bytes in 1178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:19.425985 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80b22c-8a61-480c-adea-59dcaaf88316 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:19.425985 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6a80b22c-8a61-480c-adea-59dcaaf88316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:19.466008 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7abee0eb-099e-4367-b2c1-5040c7f9518e admin admin] The token cache is being invalidate because project fe1541151785413590856ae0585ff8e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:19.471222 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7abee0eb-099e-4367-b2c1-5040c7f9518e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:19.472419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7abee0eb-099e-4367-b2c1-5040c7f9518e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:19.474141 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 900/1818] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:18 2022] DELETE /identity/v3/projects/fe1541151785413590856ae0585ff8e1 => generated 0 bytes in 975 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:19.493401 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abee0eb-099e-4367-b2c1-5040c7f9518e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:19.497473 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7abee0eb-099e-4367-b2c1-5040c7f9518e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:19.603553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80b22c-8a61-480c-adea-59dcaaf88316 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:19.673090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abee0eb-099e-4367-b2c1-5040c7f9518e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:19.692297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2a5bc-0255-4bd3-bd83-2fe4afbceb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:19.694197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2a5bc-0255-4bd3-bd83-2fe4afbceb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:19.695831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2a5bc-0255-4bd3-bd83-2fe4afbceb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:19.696625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2a5bc-0255-4bd3-bd83-2fe4afbceb51 admin admin] PATH_INFO: `/v3/users/42a1d9567cf846e0bea32ec530cbd9e4` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:19.699877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c2a5bc-0255-4bd3-bd83-2fe4afbceb51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42a1d9567cf846e0bea32ec530cbd9e4)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:19.703597 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c2a5bc-0255-4bd3-bd83-2fe4afbceb51 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:19.840524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7e361-dd32-4a66-bdb9-0c5b45795a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:19.842517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7e361-dd32-4a66-bdb9-0c5b45795a56 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:19.843200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7e361-dd32-4a66-bdb9-0c5b45795a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:19.844557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7e361-dd32-4a66-bdb9-0c5b45795a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:19.853168 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7e361-dd32-4a66-bdb9-0c5b45795a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:19.853168 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7e361-dd32-4a66-bdb9-0c5b45795a56 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:19.877838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 919/1819] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:19.883168 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161b450-3330-4200-9487-eb44ee0f96da None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:19.892114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161b450-3330-4200-9487-eb44ee0f96da None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:19.892114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161b450-3330-4200-9487-eb44ee0f96da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:19.946172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e161b450-3330-4200-9487-eb44ee0f96da None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:19.950431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e161b450-3330-4200-9487-eb44ee0f96da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:19.962419 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 920/1820] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:19.963869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161b450-3330-4200-9487-eb44ee0f96da None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:19.970169 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e161b450-3330-4200-9487-eb44ee0f96da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:19.986277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161b450-3330-4200-9487-eb44ee0f96da None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:19.992534 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d55ec0-f786-4fe1-8e75-6cc40855a20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:19.993218 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d55ec0-f786-4fe1-8e75-6cc40855a20b admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:19.999946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d55ec0-f786-4fe1-8e75-6cc40855a20b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:20.002189 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d55ec0-f786-4fe1-8e75-6cc40855a20b admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:20.003986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d55ec0-f786-4fe1-8e75-6cc40855a20b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:20.006986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d55ec0-f786-4fe1-8e75-6cc40855a20b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:20.044178 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 921/1821] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:20.045233 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4f4fb-1e9a-487b-a919-8f3ca91e09e2 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:20.045525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4f4fb-1e9a-487b-a919-8f3ca91e09e2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:20.049220 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4f4fb-1e9a-487b-a919-8f3ca91e09e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:20.063438 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-42c2a5bc-0255-4bd3-bd83-2fe4afbceb51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42a1d9567cf846e0bea32ec530cbd9e4'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:20.095968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-eeb4f4fb-1e9a-487b-a919-8f3ca91e09e2 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:20.097301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-eeb4f4fb-1e9a-487b-a919-8f3ca91e09e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:20.115706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 922/1822] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:20.116814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4f4fb-1e9a-487b-a919-8f3ca91e09e2 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:20.142102 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4f4fb-1e9a-487b-a919-8f3ca91e09e2 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:20.174861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188c115-ef10-432a-b60e-85dcfe47262f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:20.175659 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188c115-ef10-432a-b60e-85dcfe47262f admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:20.176010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188c115-ef10-432a-b60e-85dcfe47262f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:20.176418 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188c115-ef10-432a-b60e-85dcfe47262f admin admin] PATH_INFO: `/v3/projects/f3d2e282bb3948b281fb311316e5172a/users/d8929f057cc6425b9c9eb3bd2e2f7f62/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:20.186017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-42c2a5bc-0255-4bd3-bd83-2fe4afbceb51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42a1d9567cf846e0bea32ec530cbd9e4'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:20.213125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-42c2a5bc-0255-4bd3-bd83-2fe4afbceb51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42a1d9567cf846e0bea32ec530cbd9e4'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:20.214829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-42c2a5bc-0255-4bd3-bd83-2fe4afbceb51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42a1d9567cf846e0bea32ec530cbd9e4'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:20.222137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188c115-ef10-432a-b60e-85dcfe47262f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d2e282bb3948b281fb311316e5172a, user_id=d8929f057cc6425b9c9eb3bd2e2f7f62, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:20.225105 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188c115-ef10-432a-b60e-85dcfe47262f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:20.232235 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-42c2a5bc-0255-4bd3-bd83-2fe4afbceb51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42a1d9567cf846e0bea32ec530cbd9e4'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:20.249922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 901/1823] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:19 2022] DELETE /identity/v3/users/42a1d9567cf846e0bea32ec530cbd9e4 => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:20.255682 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2a5bc-0255-4bd3-bd83-2fe4afbceb51 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:20.259900 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-42c2a5bc-0255-4bd3-bd83-2fe4afbceb51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:20.368052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 923/1824] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:20 2022] PUT /identity/v3/projects/f3d2e282bb3948b281fb311316e5172a/users/d8929f057cc6425b9c9eb3bd2e2f7f62/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:20.379833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab18af-1749-46ed-8cef-d51a24eda1d4 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:20.380577 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab18af-1749-46ed-8cef-d51a24eda1d4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:20.381058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab18af-1749-46ed-8cef-d51a24eda1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:20.399702 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:20.422382 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:20.459426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d7ab18af-1749-46ed-8cef-d51a24eda1d4 None None] MFA Rules not processed for user `d8929f057cc6425b9c9eb3bd2e2f7f62`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:20.461556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d7ab18af-1749-46ed-8cef-d51a24eda1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:20.486324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2a5bc-0255-4bd3-bd83-2fe4afbceb51 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:20.645075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 924/1825] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:20.646428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b335854-bbf0-4c02-bf3d-e6dbd947d898 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:20.646763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b335854-bbf0-4c02-bf3d-e6dbd947d898 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:20.647092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b335854-bbf0-4c02-bf3d-e6dbd947d898 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:20.648664 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 925/1826] 10.222.0.91 () {54 vars in 921 bytes} [Thu Sep 8 14:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:00:20.652498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b335854-bbf0-4c02-bf3d-e6dbd947d898 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:20.658226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8b335854-bbf0-4c02-bf3d-e6dbd947d898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:20.666229 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a2610-b1b8-4444-8a7d-27af6a9db1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:20.667794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a2610-b1b8-4444-8a7d-27af6a9db1f0 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:20.668207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a2610-b1b8-4444-8a7d-27af6a9db1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:20.668730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a2610-b1b8-4444-8a7d-27af6a9db1f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:20.669695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a2610-b1b8-4444-8a7d-27af6a9db1f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:00:20.670421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a2610-b1b8-4444-8a7d-27af6a9db1f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:20.674313 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a2610-b1b8-4444-8a7d-27af6a9db1f0 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:20.694610 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 902/1827] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:00:20.696075 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a2610-b1b8-4444-8a7d-27af6a9db1f0 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:20.698446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b335854-bbf0-4c02-bf3d-e6dbd947d898 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:20.706541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65d166-3860-46c5-82a0-2233ebbab2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:20.707193 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65d166-3860-46c5-82a0-2233ebbab2bd admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:20.707471 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65d166-3860-46c5-82a0-2233ebbab2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:20.708401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65d166-3860-46c5-82a0-2233ebbab2bd admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:20.708524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65d166-3860-46c5-82a0-2233ebbab2bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:20.712297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65d166-3860-46c5-82a0-2233ebbab2bd admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:20.714751 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a2610-b1b8-4444-8a7d-27af6a9db1f0 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:20.721672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869cbd3-0d83-452e-8262-2b86a4abe82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:20.725216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869cbd3-0d83-452e-8262-2b86a4abe82f admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:20.725216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869cbd3-0d83-452e-8262-2b86a4abe82f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:20.725216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869cbd3-0d83-452e-8262-2b86a4abe82f admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:20.725216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869cbd3-0d83-452e-8262-2b86a4abe82f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:20.726447 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869cbd3-0d83-452e-8262-2b86a4abe82f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:20.968079 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 926/1828] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:20 2022] POST /identity/v3/projects => generated 375 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:20.969594 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd68014-f740-40a3-9d75-9a6ae4e9a3a7 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:20.969934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd68014-f740-40a3-9d75-9a6ae4e9a3a7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:20.970215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd68014-f740-40a3-9d75-9a6ae4e9a3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:20.985228 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 903/1829] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:20 2022] POST /identity/v3/projects => generated 381 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:20.986788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869cbd3-0d83-452e-8262-2b86a4abe82f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:21.007011 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-acd68014-f740-40a3-9d75-9a6ae4e9a3a7 None None] MFA Rules not processed for user `38127f41852d412ea6ad270f1618fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:21.008865 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-acd68014-f740-40a3-9d75-9a6ae4e9a3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:21.145184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 927/1830] 10.222.0.91 () {58 vars in 1018 bytes} [Thu Sep 8 14:00:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:21.147345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd68014-f740-40a3-9d75-9a6ae4e9a3a7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:21.175830 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869cbd3-0d83-452e-8262-2b86a4abe82f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:21.183902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779dcb9-07f4-4d23-a8e9-2e37bb6b887a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:21.185525 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779dcb9-07f4-4d23-a8e9-2e37bb6b887a admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:21.185970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779dcb9-07f4-4d23-a8e9-2e37bb6b887a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:21.186643 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779dcb9-07f4-4d23-a8e9-2e37bb6b887a admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:21.187906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779dcb9-07f4-4d23-a8e9-2e37bb6b887a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:21.192292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779dcb9-07f4-4d23-a8e9-2e37bb6b887a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:21.313552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 904/1831] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:20 2022] POST /identity/v3/users => generated 294 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:21.313552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777779db-5180-4974-84f1-bd18287a685e None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:21.313552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777779db-5180-4974-84f1-bd18287a685e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:21.313552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777779db-5180-4974-84f1-bd18287a685e None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:21.315324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 905/1832] 10.222.0.91 () {54 vars in 921 bytes} [Thu Sep 8 14:00:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:00:21.316747 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777779db-5180-4974-84f1-bd18287a685e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:21.345909 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777779db-5180-4974-84f1-bd18287a685e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:21.353366 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa0056-a629-4bff-bdf6-ddf61963ae8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:21.354225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa0056-a629-4bff-bdf6-ddf61963ae8a admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:21.354656 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa0056-a629-4bff-bdf6-ddf61963ae8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:21.355084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa0056-a629-4bff-bdf6-ddf61963ae8a admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:21.356053 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa0056-a629-4bff-bdf6-ddf61963ae8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:21.370285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa0056-a629-4bff-bdf6-ddf61963ae8a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:21.417498 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 906/1833] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:21.419759 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa0056-a629-4bff-bdf6-ddf61963ae8a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:21.465481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd68014-f740-40a3-9d75-9a6ae4e9a3a7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:21.487282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa0056-a629-4bff-bdf6-ddf61963ae8a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:21.496790 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87176fd1-b883-44e8-9bb6-b135f2b82c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:21.498653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87176fd1-b883-44e8-9bb6-b135f2b82c9e admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:21.499032 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87176fd1-b883-44e8-9bb6-b135f2b82c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:21.499993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87176fd1-b883-44e8-9bb6-b135f2b82c9e admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:21.503426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87176fd1-b883-44e8-9bb6-b135f2b82c9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:21.507921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84deefe-cdf9-4879-951d-a903529002f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:21.507921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84deefe-cdf9-4879-951d-a903529002f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:21.507921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84deefe-cdf9-4879-951d-a903529002f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:21.507921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84deefe-cdf9-4879-951d-a903529002f5 admin admin] PATH_INFO: `/v3/projects/20b10de4152748169f1e7d3246d2a883/users/e8f60f42606b4c49a6a9194a1545f9cb/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:21.508453 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87176fd1-b883-44e8-9bb6-b135f2b82c9e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:21.544357 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84deefe-cdf9-4879-951d-a903529002f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20b10de4152748169f1e7d3246d2a883, user_id=e8f60f42606b4c49a6a9194a1545f9cb, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:21.548152 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84deefe-cdf9-4879-951d-a903529002f5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:21.671965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 907/1835] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:21 2022] PUT /identity/v3/projects/20b10de4152748169f1e7d3246d2a883/users/e8f60f42606b4c49a6a9194a1545f9cb/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:21.672805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 928/1835] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:21 2022] POST /identity/v3/users => generated 291 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:21.672805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87176fd1-b883-44e8-9bb6-b135f2b82c9e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:21.672993 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af782b42-64a5-415d-9a53-277c141c7002 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:21.672993 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af782b42-64a5-415d-9a53-277c141c7002 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:21.672993 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af782b42-64a5-415d-9a53-277c141c7002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:21.710844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87176fd1-b883-44e8-9bb6-b135f2b82c9e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:21.724345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-af782b42-64a5-415d-9a53-277c141c7002 None None] MFA Rules not processed for user `38127f41852d412ea6ad270f1618fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:21.725634 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-af782b42-64a5-415d-9a53-277c141c7002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:21.980036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 908/1836] 10.222.0.91 () {58 vars in 1018 bytes} [Thu Sep 8 14:00:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:21.982339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af782b42-64a5-415d-9a53-277c141c7002 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:22.024101 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277e4cc-d703-42a5-99bf-8a04c4c1434d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:22.024754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277e4cc-d703-42a5-99bf-8a04c4c1434d admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:22.025448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277e4cc-d703-42a5-99bf-8a04c4c1434d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:22.025536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277e4cc-d703-42a5-99bf-8a04c4c1434d admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:22.028549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277e4cc-d703-42a5-99bf-8a04c4c1434d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:22.040078 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277e4cc-d703-42a5-99bf-8a04c4c1434d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:22.045993 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:22.089696 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 929/1837] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:22.098833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277e4cc-d703-42a5-99bf-8a04c4c1434d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:22.118878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277e4cc-d703-42a5-99bf-8a04c4c1434d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:22.128503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f7e18-5dbd-4f2c-a9ca-4b2b953efa4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:22.132842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f7e18-5dbd-4f2c-a9ca-4b2b953efa4c admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:22.133440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f7e18-5dbd-4f2c-a9ca-4b2b953efa4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:22.134776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f7e18-5dbd-4f2c-a9ca-4b2b953efa4c admin admin] PATH_INFO: `/v3/projects/20b10de4152748169f1e7d3246d2a883/users/e8f60f42606b4c49a6a9194a1545f9cb/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:22.143612 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f7e18-5dbd-4f2c-a9ca-4b2b953efa4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=20b10de4152748169f1e7d3246d2a883, user_id=e8f60f42606b4c49a6a9194a1545f9cb, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:22.149658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f7e18-5dbd-4f2c-a9ca-4b2b953efa4c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:22.166035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af782b42-64a5-415d-9a53-277c141c7002 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:22.173108 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a1fc8-bb46-4ebf-99ba-d3a82f74b916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:22.181722 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a1fc8-bb46-4ebf-99ba-d3a82f74b916 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:22.182869 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a1fc8-bb46-4ebf-99ba-d3a82f74b916 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:22.183523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a1fc8-bb46-4ebf-99ba-d3a82f74b916 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:22.184678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a1fc8-bb46-4ebf-99ba-d3a82f74b916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:22.186748 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a1fc8-bb46-4ebf-99ba-d3a82f74b916 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:22.200912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 909/1838] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:21 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:22.201110 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 930/1839] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:22 2022] PUT /identity/v3/projects/20b10de4152748169f1e7d3246d2a883/users/e8f60f42606b4c49a6a9194a1545f9cb/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:22.205775 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f7e18-5dbd-4f2c-a9ca-4b2b953efa4c admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:22.214188 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923df8f-2c83-48a2-8cd2-cafefce14ae8 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:22.214476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923df8f-2c83-48a2-8cd2-cafefce14ae8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:22.214869 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923df8f-2c83-48a2-8cd2-cafefce14ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:22.274394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d923df8f-2c83-48a2-8cd2-cafefce14ae8 None None] MFA Rules not processed for user `e8f60f42606b4c49a6a9194a1545f9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:22.276540 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d923df8f-2c83-48a2-8cd2-cafefce14ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:22.404855 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f7e18-5dbd-4f2c-a9ca-4b2b953efa4c admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:22.417568 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15930ffe-deae-41fe-ab31-ae2054e1add0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:22.420160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15930ffe-deae-41fe-ab31-ae2054e1add0 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:22.421203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15930ffe-deae-41fe-ab31-ae2054e1add0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:22.421203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15930ffe-deae-41fe-ab31-ae2054e1add0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:22.422010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15930ffe-deae-41fe-ab31-ae2054e1add0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:22.425379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15930ffe-deae-41fe-ab31-ae2054e1add0 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:22.449569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 931/1840] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:22.449569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15930ffe-deae-41fe-ab31-ae2054e1add0 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:22.452271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-15930ffe-deae-41fe-ab31-ae2054e1add0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:22.482138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 910/1841] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:22.483664 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923df8f-2c83-48a2-8cd2-cafefce14ae8 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:22.509761 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923df8f-2c83-48a2-8cd2-cafefce14ae8 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:22.519624 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91927f7b-c6b7-4a0c-b24f-17440f6a5190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:22.520672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91927f7b-c6b7-4a0c-b24f-17440f6a5190 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:22.521161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91927f7b-c6b7-4a0c-b24f-17440f6a5190 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:22.521693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91927f7b-c6b7-4a0c-b24f-17440f6a5190 admin admin] PATH_INFO: `/v3/projects/7e5a739f6dc74f06888e0ae3be9378da/users/a3fd73eafe5b46f49a7488f478245b96/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:22.570196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91927f7b-c6b7-4a0c-b24f-17440f6a5190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e5a739f6dc74f06888e0ae3be9378da, user_id=a3fd73eafe5b46f49a7488f478245b96, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:22.581218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91927f7b-c6b7-4a0c-b24f-17440f6a5190 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:22.734748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15930ffe-deae-41fe-ab31-ae2054e1add0 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:22.750878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d826f6-dc0d-4e0c-b44d-e119b99870a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:22.751269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d826f6-dc0d-4e0c-b44d-e119b99870a4 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:22.751631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d826f6-dc0d-4e0c-b44d-e119b99870a4 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:22.752011 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d826f6-dc0d-4e0c-b44d-e119b99870a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:22.757322 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 911/1842] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:22 2022] PUT /identity/v3/projects/7e5a739f6dc74f06888e0ae3be9378da/users/a3fd73eafe5b46f49a7488f478245b96/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:22.759524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91927f7b-c6b7-4a0c-b24f-17440f6a5190 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:22.773065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-91927f7b-c6b7-4a0c-b24f-17440f6a5190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:22.786700 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d826f6-dc0d-4e0c-b44d-e119b99870a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:22.790197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d826f6-dc0d-4e0c-b44d-e119b99870a4 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:22.830565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 932/1843] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:22.836888 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed2446-6a73-43df-a9e1-d195653d5844 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:22.837232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed2446-6a73-43df-a9e1-d195653d5844 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:22.837437 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed2446-6a73-43df-a9e1-d195653d5844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:22.877969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91927f7b-c6b7-4a0c-b24f-17440f6a5190 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:22.887407 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a05905-383d-41a3-a728-2d9816de732d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:22.888047 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a05905-383d-41a3-a728-2d9816de732d service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:22.888327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a05905-383d-41a3-a728-2d9816de732d service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:22.888593 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a05905-383d-41a3-a728-2d9816de732d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:22.942267 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-61ed2446-6a73-43df-a9e1-d195653d5844 None None] MFA Rules not processed for user `a3fd73eafe5b46f49a7488f478245b96`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:22.944126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-61ed2446-6a73-43df-a9e1-d195653d5844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:23.040938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a05905-383d-41a3-a728-2d9816de732d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:23.040938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a05905-383d-41a3-a728-2d9816de732d service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:23.067515 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 912/1844] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:23.149415 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a05905-383d-41a3-a728-2d9816de732d service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:23.153107 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-67a05905-383d-41a3-a728-2d9816de732d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:23.190521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 933/1845] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:23.195704 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed2446-6a73-43df-a9e1-d195653d5844 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:23.344005 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:23.349380 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed2446-6a73-43df-a9e1-d195653d5844 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:23.361001 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ad100-fc68-4282-a2ea-7fc519f2708a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:23.362335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ad100-fc68-4282-a2ea-7fc519f2708a admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:23.363271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ad100-fc68-4282-a2ea-7fc519f2708a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:23.363811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ad100-fc68-4282-a2ea-7fc519f2708a admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:23.364849 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ad100-fc68-4282-a2ea-7fc519f2708a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:23.372927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ad100-fc68-4282-a2ea-7fc519f2708a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:23.379680 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a05905-383d-41a3-a728-2d9816de732d service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:23.387423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa7f5a-c5c8-4ee4-97bb-b084f91905be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:23.388154 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa7f5a-c5c8-4ee4-97bb-b084f91905be service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:23.388500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa7f5a-c5c8-4ee4-97bb-b084f91905be service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:23.388951 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa7f5a-c5c8-4ee4-97bb-b084f91905be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:23.406907 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa7f5a-c5c8-4ee4-97bb-b084f91905be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:23.409566 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aa7f5a-c5c8-4ee4-97bb-b084f91905be service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:23.440500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 913/1846] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:00:23.522723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 934/1847] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:23 2022] POST /identity/v3/projects => generated 379 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:23.522829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa7f5a-c5c8-4ee4-97bb-b084f91905be service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:23.540039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa7f5a-c5c8-4ee4-97bb-b084f91905be service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:23.710844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6b163-a594-4a1b-b2a8-f462b1231bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:23.712555 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6b163-a594-4a1b-b2a8-f462b1231bda admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:23.712945 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6b163-a594-4a1b-b2a8-f462b1231bda admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:23.713908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6b163-a594-4a1b-b2a8-f462b1231bda admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:23.714767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6b163-a594-4a1b-b2a8-f462b1231bda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:23.729677 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6b163-a594-4a1b-b2a8-f462b1231bda admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:23.879541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ad100-fc68-4282-a2ea-7fc519f2708a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:23.881341 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d23ad100-fc68-4282-a2ea-7fc519f2708a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:23.895285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 914/1848] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:23 2022] POST /identity/v3/users => generated 296 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:23.898951 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6b163-a594-4a1b-b2a8-f462b1231bda admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:23.905540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ad100-fc68-4282-a2ea-7fc519f2708a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:23.918635 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3ff58-4a73-4d62-a34a-5c0a79ff18f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:23.920130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3ff58-4a73-4d62-a34a-5c0a79ff18f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:23.920428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3ff58-4a73-4d62-a34a-5c0a79ff18f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:23.920680 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3ff58-4a73-4d62-a34a-5c0a79ff18f6 admin admin] PATH_INFO: `/v3/projects/86afe3ee752f481d90dfffb470872f1b` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:23.922724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3ff58-4a73-4d62-a34a-5c0a79ff18f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86afe3ee752f481d90dfffb470872f1b)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:23.925817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3ff58-4a73-4d62-a34a-5c0a79ff18f6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:23.927049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6b163-a594-4a1b-b2a8-f462b1231bda admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:23.947061 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed542262-f737-499e-84f4-7912aa0335c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:23.948400 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed542262-f737-499e-84f4-7912aa0335c4 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:23.948780 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed542262-f737-499e-84f4-7912aa0335c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:23.949221 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed542262-f737-499e-84f4-7912aa0335c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:23.950055 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed542262-f737-499e-84f4-7912aa0335c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:23.952567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed542262-f737-499e-84f4-7912aa0335c4 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:23.983122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 915/1849] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:23.987479 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed542262-f737-499e-84f4-7912aa0335c4 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:24.012912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed542262-f737-499e-84f4-7912aa0335c4 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:24.033943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4f883-95bb-4ca8-abf7-00f6d55d6ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:24.036048 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4f883-95bb-4ca8-abf7-00f6d55d6ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:24.036430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4f883-95bb-4ca8-abf7-00f6d55d6ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:24.037209 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4f883-95bb-4ca8-abf7-00f6d55d6ad2 admin admin] PATH_INFO: `/v3/projects/a4bfda412caa4edc8c3d5112c685e72a/users/8055ed7a4d684c96b800c35497b3dbc7/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:24.103518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4f883-95bb-4ca8-abf7-00f6d55d6ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4bfda412caa4edc8c3d5112c685e72a, user_id=8055ed7a4d684c96b800c35497b3dbc7, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:24.108665 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4f883-95bb-4ca8-abf7-00f6d55d6ad2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:24.255894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 916/1850] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:23 2022] PUT /identity/v3/projects/a4bfda412caa4edc8c3d5112c685e72a/users/8055ed7a4d684c96b800c35497b3dbc7/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:24.258586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4f883-95bb-4ca8-abf7-00f6d55d6ad2 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:24.293242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ee3ff58-4a73-4d62-a34a-5c0a79ff18f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86afe3ee752f481d90dfffb470872f1b', 'request_id': 'req-2ee3ff58-4a73-4d62-a34a-5c0a79ff18f6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:24.426930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4f883-95bb-4ca8-abf7-00f6d55d6ad2 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:24.443259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ee3ff58-4a73-4d62-a34a-5c0a79ff18f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86afe3ee752f481d90dfffb470872f1b', 'request_id': 'req-2ee3ff58-4a73-4d62-a34a-5c0a79ff18f6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:24.460122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3faf01-549a-4c3e-a20f-778d322796a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:24.460122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3faf01-549a-4c3e-a20f-778d322796a5 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:24.460122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3faf01-549a-4c3e-a20f-778d322796a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:24.460122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3faf01-549a-4c3e-a20f-778d322796a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:24.460622 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3faf01-549a-4c3e-a20f-778d322796a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:24.460622 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3faf01-549a-4c3e-a20f-778d322796a5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:24.460777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ee3ff58-4a73-4d62-a34a-5c0a79ff18f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86afe3ee752f481d90dfffb470872f1b', 'request_id': 'req-2ee3ff58-4a73-4d62-a34a-5c0a79ff18f6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:24.493979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 917/1851] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:24.499736 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3faf01-549a-4c3e-a20f-778d322796a5 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:24.512641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ee3ff58-4a73-4d62-a34a-5c0a79ff18f6 admin admin] The token cache is being invalidate because project 86afe3ee752f481d90dfffb470872f1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:24.513725 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ee3ff58-4a73-4d62-a34a-5c0a79ff18f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:24.514654 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3faf01-549a-4c3e-a20f-778d322796a5 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:24.515440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ee3ff58-4a73-4d62-a34a-5c0a79ff18f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:24.518490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 935/1852] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:23 2022] DELETE /identity/v3/projects/86afe3ee752f481d90dfffb470872f1b => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:24.525919 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce907e-4a31-41ea-96b0-640abd39b0dc None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:24.527061 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce907e-4a31-41ea-96b0-640abd39b0dc None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:24.527864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce907e-4a31-41ea-96b0-640abd39b0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:24.529231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fc1c8-5557-4f2c-a1e8-bbafea2094bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:24.529231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fc1c8-5557-4f2c-a1e8-bbafea2094bb admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:24.529231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fc1c8-5557-4f2c-a1e8-bbafea2094bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:24.529800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fc1c8-5557-4f2c-a1e8-bbafea2094bb admin admin] PATH_INFO: `/v3/projects/a4bfda412caa4edc8c3d5112c685e72a/users/8055ed7a4d684c96b800c35497b3dbc7/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:24.533179 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fc1c8-5557-4f2c-a1e8-bbafea2094bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4bfda412caa4edc8c3d5112c685e72a, user_id=8055ed7a4d684c96b800c35497b3dbc7, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:24.536506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fc1c8-5557-4f2c-a1e8-bbafea2094bb admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:24.556585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 918/1853] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:24 2022] PUT /identity/v3/projects/a4bfda412caa4edc8c3d5112c685e72a/users/8055ed7a4d684c96b800c35497b3dbc7/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:24.571274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-5dce907e-4a31-41ea-96b0-640abd39b0dc None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:24.573346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5dce907e-4a31-41ea-96b0-640abd39b0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:24.575308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fe5ea-dc1f-4f3c-bf18-7dcbac6c9c47 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:24.575922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fe5ea-dc1f-4f3c-bf18-7dcbac6c9c47 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:24.576410 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fe5ea-dc1f-4f3c-bf18-7dcbac6c9c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:24.692911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-959fe5ea-dc1f-4f3c-bf18-7dcbac6c9c47 None None] MFA Rules not processed for user `8055ed7a4d684c96b800c35497b3dbc7`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:24.694716 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-959fe5ea-dc1f-4f3c-bf18-7dcbac6c9c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:24.752729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 936/1854] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:24.764819 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e329e-2910-45c0-b60e-7279a360ac99 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:24.769208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e329e-2910-45c0-b60e-7279a360ac99 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:24.771157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e329e-2910-45c0-b60e-7279a360ac99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:24.813770 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3a1e329e-2910-45c0-b60e-7279a360ac99 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:24.815448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3a1e329e-2910-45c0-b60e-7279a360ac99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:24.853305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 937/1855] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:24.856581 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e329e-2910-45c0-b60e-7279a360ac99 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:24.879669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e329e-2910-45c0-b60e-7279a360ac99 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:24.886031 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5ac1c-3da1-4e3a-b387-6ad2b3757cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:24.886778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5ac1c-3da1-4e3a-b387-6ad2b3757cea admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:24.887259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5ac1c-3da1-4e3a-b387-6ad2b3757cea admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:24.888560 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5ac1c-3da1-4e3a-b387-6ad2b3757cea admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:24.889696 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5ac1c-3da1-4e3a-b387-6ad2b3757cea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:00:24.890234 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5ac1c-3da1-4e3a-b387-6ad2b3757cea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:24.893219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5ac1c-3da1-4e3a-b387-6ad2b3757cea admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:24.922780 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 919/1856] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:24 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:24.927939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 938/1857] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:00:24.929146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fe5ea-dc1f-4f3c-bf18-7dcbac6c9c47 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:24.940806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5ac1c-3da1-4e3a-b387-6ad2b3757cea admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:24.945385 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-77d5ac1c-3da1-4e3a-b387-6ad2b3757cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:24.971132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:24.972510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fe5ea-dc1f-4f3c-bf18-7dcbac6c9c47 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:24.978558 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4524a-a5e9-4a4e-aa34-8aed0d39e146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:24.979620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4524a-a5e9-4a4e-aa34-8aed0d39e146 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:24.980128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4524a-a5e9-4a4e-aa34-8aed0d39e146 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:24.980746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b4524a-a5e9-4a4e-aa34-8aed0d39e146 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:24.982020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4524a-a5e9-4a4e-aa34-8aed0d39e146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:24.984584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b4524a-a5e9-4a4e-aa34-8aed0d39e146 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:25.067351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5ac1c-3da1-4e3a-b387-6ad2b3757cea admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:25.073869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8a04e-24ea-4510-beeb-ecd0d110684b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:25.074506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8a04e-24ea-4510-beeb-ecd0d110684b service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:25.074823 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8a04e-24ea-4510-beeb-ecd0d110684b service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:25.075116 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8a04e-24ea-4510-beeb-ecd0d110684b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:25.089409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8a04e-24ea-4510-beeb-ecd0d110684b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:25.091085 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8a04e-24ea-4510-beeb-ecd0d110684b service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:25.111662 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 939/1858] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:25.226976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 920/1859] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:24 2022] POST /identity/v3/projects => generated 367 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:25.228314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8a04e-24ea-4510-beeb-ecd0d110684b service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:25.268400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8a04e-24ea-4510-beeb-ecd0d110684b service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:25.625069 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7cf57-d77f-4c42-be28-4a5926b4c9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:25.626069 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7cf57-d77f-4c42-be28-4a5926b4c9ee admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:25.626554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7cf57-d77f-4c42-be28-4a5926b4c9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:25.627097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7cf57-d77f-4c42-be28-4a5926b4c9ee admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:25.628569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7cf57-d77f-4c42-be28-4a5926b4c9ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:25.637253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7cf57-d77f-4c42-be28-4a5926b4c9ee admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:25.833827 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 940/1860] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:25 2022] POST /identity/v3/users => generated 284 bytes in 606 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:25.847880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4524a-a5e9-4a4e-aa34-8aed0d39e146 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:25.880741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b4524a-a5e9-4a4e-aa34-8aed0d39e146 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:25.911266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48779f-691e-4156-b10c-1cb64fcbc29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:25.911266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48779f-691e-4156-b10c-1cb64fcbc29f admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:25.911266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48779f-691e-4156-b10c-1cb64fcbc29f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:25.911266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48779f-691e-4156-b10c-1cb64fcbc29f admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:25.911914 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48779f-691e-4156-b10c-1cb64fcbc29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:25.911914 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48779f-691e-4156-b10c-1cb64fcbc29f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:25.951401 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 921/1861] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:25.955260 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7cf57-d77f-4c42-be28-4a5926b4c9ee admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:25.989949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7cf57-d77f-4c42-be28-4a5926b4c9ee admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:26.038356 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7deb4d-973b-42b9-843b-061b48c8006d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:26.039908 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7deb4d-973b-42b9-843b-061b48c8006d admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:26.040486 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7deb4d-973b-42b9-843b-061b48c8006d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:26.041224 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7deb4d-973b-42b9-843b-061b48c8006d admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:26.045394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7deb4d-973b-42b9-843b-061b48c8006d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:26.048565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7deb4d-973b-42b9-843b-061b48c8006d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:26.161597 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48779f-691e-4156-b10c-1cb64fcbc29f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:26.174922 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 941/1862] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:26.192108 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48779f-691e-4156-b10c-1cb64fcbc29f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:26.200805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3dea7-6ac4-44bc-baff-4788e62a4a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:26.202742 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3dea7-6ac4-44bc-baff-4788e62a4a5e admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:26.202742 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3dea7-6ac4-44bc-baff-4788e62a4a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:26.202742 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3dea7-6ac4-44bc-baff-4788e62a4a5e admin admin] PATH_INFO: `/v3/projects/e5d8409e5cc34d1aa04009d96c988496/users/fb81ae95248b48c18d5398fea534c8fa/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:26.247067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3dea7-6ac4-44bc-baff-4788e62a4a5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d8409e5cc34d1aa04009d96c988496, user_id=fb81ae95248b48c18d5398fea534c8fa, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:26.256077 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3dea7-6ac4-44bc-baff-4788e62a4a5e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:26.475579 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 922/1863] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:26 2022] PUT /identity/v3/projects/e5d8409e5cc34d1aa04009d96c988496/users/fb81ae95248b48c18d5398fea534c8fa/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:26.486571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631b005-ae56-42dc-92f8-4e8284d94663 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:26.486861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631b005-ae56-42dc-92f8-4e8284d94663 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:26.487341 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631b005-ae56-42dc-92f8-4e8284d94663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:26.562617 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-7631b005-ae56-42dc-92f8-4e8284d94663 None None] MFA Rules not processed for user `fb81ae95248b48c18d5398fea534c8fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:26.562617 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7631b005-ae56-42dc-92f8-4e8284d94663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:26.742080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3dea7-6ac4-44bc-baff-4788e62a4a5e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:26.750237 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 942/1864] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:26.831352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:26.883000 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3dea7-6ac4-44bc-baff-4788e62a4a5e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:26.891387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5e007-72c1-4a9c-ba7d-b32f11823b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:26.898915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5e007-72c1-4a9c-ba7d-b32f11823b75 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:26.898915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5e007-72c1-4a9c-ba7d-b32f11823b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:26.898915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5e007-72c1-4a9c-ba7d-b32f11823b75 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:26.898915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5e007-72c1-4a9c-ba7d-b32f11823b75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:26.898915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5e007-72c1-4a9c-ba7d-b32f11823b75 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:26.976212 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:27.114586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 923/1865] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:26 2022] POST /identity/v3/projects => generated 369 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:00:27.118832 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631b005-ae56-42dc-92f8-4e8284d94663 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:27.135268 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631b005-ae56-42dc-92f8-4e8284d94663 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:27.191579 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5e007-72c1-4a9c-ba7d-b32f11823b75 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:27.196712 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-20e5e007-72c1-4a9c-ba7d-b32f11823b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:27.231037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d9309-69bb-4fd9-932f-b2f1c7a4f87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:27.231725 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d9309-69bb-4fd9-932f-b2f1c7a4f87d admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:27.232139 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d9309-69bb-4fd9-932f-b2f1c7a4f87d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:27.232487 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d9309-69bb-4fd9-932f-b2f1c7a4f87d admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:27.233183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d9309-69bb-4fd9-932f-b2f1c7a4f87d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:27.235385 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d9309-69bb-4fd9-932f-b2f1c7a4f87d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:27.279094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5e007-72c1-4a9c-ba7d-b32f11823b75 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:27.285551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd35907-4417-4a8f-8c74-a453d6b2c7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:27.286194 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd35907-4417-4a8f-8c74-a453d6b2c7db admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:27.286474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd35907-4417-4a8f-8c74-a453d6b2c7db admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:27.286745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd35907-4417-4a8f-8c74-a453d6b2c7db admin admin] PATH_INFO: `/v3/users/a3fd73eafe5b46f49a7488f478245b96` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:27.291925 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd35907-4417-4a8f-8c74-a453d6b2c7db admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3fd73eafe5b46f49a7488f478245b96)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:27.294232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd35907-4417-4a8f-8c74-a453d6b2c7db admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:27.352189 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 943/1866] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:27 2022] POST /identity/v3/users => generated 291 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:27.357530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d9309-69bb-4fd9-932f-b2f1c7a4f87d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:27.369613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d9309-69bb-4fd9-932f-b2f1c7a4f87d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:27.374380 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff04033-56bf-4b96-811f-9224837f8b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:27.375088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff04033-56bf-4b96-811f-9224837f8b17 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:27.375442 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff04033-56bf-4b96-811f-9224837f8b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:27.375792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff04033-56bf-4b96-811f-9224837f8b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:27.376536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff04033-56bf-4b96-811f-9224837f8b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:27.378439 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff04033-56bf-4b96-811f-9224837f8b17 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:27.388872 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 944/1867] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:27.394912 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff04033-56bf-4b96-811f-9224837f8b17 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:27.405953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff04033-56bf-4b96-811f-9224837f8b17 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:27.410738 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce37f5-666c-473e-9dcc-4129e1003b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:27.411441 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce37f5-666c-473e-9dcc-4129e1003b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:27.411725 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce37f5-666c-473e-9dcc-4129e1003b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:27.412094 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce37f5-666c-473e-9dcc-4129e1003b6a admin admin] PATH_INFO: `/v3/projects/2a235c37c279428dad3c2fca4a1cc0df/users/4374731ce88c4be4ac984f38c2ebb9ab/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:27.429260 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce37f5-666c-473e-9dcc-4129e1003b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a235c37c279428dad3c2fca4a1cc0df, user_id=4374731ce88c4be4ac984f38c2ebb9ab, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:27.432006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce37f5-666c-473e-9dcc-4129e1003b6a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:27.518289 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 945/1868] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:27 2022] PUT /identity/v3/projects/2a235c37c279428dad3c2fca4a1cc0df/users/4374731ce88c4be4ac984f38c2ebb9ab/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:27.522532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce37f5-666c-473e-9dcc-4129e1003b6a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:27.609078 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce37f5-666c-473e-9dcc-4129e1003b6a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:27.616595 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe212a28-ff51-4084-8c4b-7440226cdd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:27.616595 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe212a28-ff51-4084-8c4b-7440226cdd0e admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:27.616595 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe212a28-ff51-4084-8c4b-7440226cdd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:27.625219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe212a28-ff51-4084-8c4b-7440226cdd0e admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:27.625219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe212a28-ff51-4084-8c4b-7440226cdd0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:27.625219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe212a28-ff51-4084-8c4b-7440226cdd0e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:27.636150 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 946/1869] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:00:27.639909 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bfd35907-4417-4a8f-8c74-a453d6b2c7db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3fd73eafe5b46f49a7488f478245b96'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:27.642769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe212a28-ff51-4084-8c4b-7440226cdd0e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:27.652084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe212a28-ff51-4084-8c4b-7440226cdd0e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:27.660588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b0ba9c-01ef-46a9-a8e1-b94d7f66365f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:27.661276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0ba9c-01ef-46a9-a8e1-b94d7f66365f admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:27.661576 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0ba9c-01ef-46a9-a8e1-b94d7f66365f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:27.661857 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b0ba9c-01ef-46a9-a8e1-b94d7f66365f admin admin] PATH_INFO: `/v3/projects/2a235c37c279428dad3c2fca4a1cc0df/users/4374731ce88c4be4ac984f38c2ebb9ab/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:27.663930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0ba9c-01ef-46a9-a8e1-b94d7f66365f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a235c37c279428dad3c2fca4a1cc0df, user_id=4374731ce88c4be4ac984f38c2ebb9ab, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:27.667911 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b0ba9c-01ef-46a9-a8e1-b94d7f66365f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:27.678295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 947/1870] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:27 2022] PUT /identity/v3/projects/2a235c37c279428dad3c2fca4a1cc0df/users/4374731ce88c4be4ac984f38c2ebb9ab/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:27.689894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2359f38-0e76-475a-b8e2-7a4efdd546a4 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:27.690173 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2359f38-0e76-475a-b8e2-7a4efdd546a4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:27.690444 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2359f38-0e76-475a-b8e2-7a4efdd546a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:27.723302 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d2359f38-0e76-475a-b8e2-7a4efdd546a4 None None] MFA Rules not processed for user `4374731ce88c4be4ac984f38c2ebb9ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:27.724540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d2359f38-0e76-475a-b8e2-7a4efdd546a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:27.757635 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bfd35907-4417-4a8f-8c74-a453d6b2c7db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3fd73eafe5b46f49a7488f478245b96'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:27.776428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bfd35907-4417-4a8f-8c74-a453d6b2c7db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3fd73eafe5b46f49a7488f478245b96'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:27.777512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bfd35907-4417-4a8f-8c74-a453d6b2c7db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3fd73eafe5b46f49a7488f478245b96'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:27.785390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bfd35907-4417-4a8f-8c74-a453d6b2c7db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3fd73eafe5b46f49a7488f478245b96'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:27.791965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 924/1871] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:27 2022] DELETE /identity/v3/users/a3fd73eafe5b46f49a7488f478245b96 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:27.796383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf539fc-6e68-43fd-8128-cc812e6945da None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:27.796672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf539fc-6e68-43fd-8128-cc812e6945da None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:27.796971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf539fc-6e68-43fd-8128-cc812e6945da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:27.798615 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 948/1872] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:27.810260 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2359f38-0e76-475a-b8e2-7a4efdd546a4 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:27.811944 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d2359f38-0e76-475a-b8e2-7a4efdd546a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:27.817712 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-bbf539fc-6e68-43fd-8128-cc812e6945da None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:27.819072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bbf539fc-6e68-43fd-8128-cc812e6945da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:27.878200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2359f38-0e76-475a-b8e2-7a4efdd546a4 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:27.883413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 925/1873] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:27.894632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf539fc-6e68-43fd-8128-cc812e6945da None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:27.896109 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bbf539fc-6e68-43fd-8128-cc812e6945da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:27.933648 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50873a2-b100-4ce5-ae7e-9e645a4ce84b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:27.934327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50873a2-b100-4ce5-ae7e-9e645a4ce84b service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:27.934606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50873a2-b100-4ce5-ae7e-9e645a4ce84b service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:27.934882 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50873a2-b100-4ce5-ae7e-9e645a4ce84b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:27.962741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf539fc-6e68-43fd-8128-cc812e6945da None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:27.975602 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aba3f1-f9d9-4ef4-b5eb-26603add55fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:27.976342 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aba3f1-f9d9-4ef4-b5eb-26603add55fa service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:27.976913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aba3f1-f9d9-4ef4-b5eb-26603add55fa service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:27.976913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aba3f1-f9d9-4ef4-b5eb-26603add55fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:27.990043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aba3f1-f9d9-4ef4-b5eb-26603add55fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:27.991813 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aba3f1-f9d9-4ef4-b5eb-26603add55fa service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:28.002326 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50873a2-b100-4ce5-ae7e-9e645a4ce84b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:28.003842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50873a2-b100-4ce5-ae7e-9e645a4ce84b service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:28.004844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 926/1874] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:00:28.014994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 949/1875] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:00:28.138932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aba3f1-f9d9-4ef4-b5eb-26603add55fa service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:28.141528 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-42aba3f1-f9d9-4ef4-b5eb-26603add55fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:28.154292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aba3f1-f9d9-4ef4-b5eb-26603add55fa service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:28.160325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f2496-f001-4a5e-9c35-0e7132559995 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:28.160977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f2496-f001-4a5e-9c35-0e7132559995 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:28.161281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f2496-f001-4a5e-9c35-0e7132559995 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:28.161554 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f2496-f001-4a5e-9c35-0e7132559995 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:28.163409 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3c8f2496-f001-4a5e-9c35-0e7132559995 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:28.247634 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f2496-f001-4a5e-9c35-0e7132559995 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:28.248738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f2496-f001-4a5e-9c35-0e7132559995 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:28.261291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 927/1876] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:28.611536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50873a2-b100-4ce5-ae7e-9e645a4ce84b service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:28.613170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e50873a2-b100-4ce5-ae7e-9e645a4ce84b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:28.628253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50873a2-b100-4ce5-ae7e-9e645a4ce84b service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:28.633791 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:28.634438 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:28.634742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:28.635078 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5 admin admin] PATH_INFO: `/v3/projects/7e5a739f6dc74f06888e0ae3be9378da` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:28.636763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e5a739f6dc74f06888e0ae3be9378da)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:28.638756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:28.831161 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e5a739f6dc74f06888e0ae3be9378da', 'request_id': 'req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:29.027562 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e5a739f6dc74f06888e0ae3be9378da', 'request_id': 'req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:29.034790 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e5a739f6dc74f06888e0ae3be9378da', 'request_id': 'req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:29.046833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5 admin admin] The token cache is being invalidate because project 7e5a739f6dc74f06888e0ae3be9378da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:29.047147 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:29.047975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:29.049637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 950/1877] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:28 2022] DELETE /identity/v3/projects/7e5a739f6dc74f06888e0ae3be9378da => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:29.054051 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f2496-f001-4a5e-9c35-0e7132559995 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:29.064862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f2496-f001-4a5e-9c35-0e7132559995 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:29.121213 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab82d7-05e9-4c14-8bd2-73f4171c42a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:29.121933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab82d7-05e9-4c14-8bd2-73f4171c42a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:29.122209 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab82d7-05e9-4c14-8bd2-73f4171c42a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:29.122481 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab82d7-05e9-4c14-8bd2-73f4171c42a1 admin admin] PATH_INFO: `/v3/users/8055ed7a4d684c96b800c35497b3dbc7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:29.123626 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab82d7-05e9-4c14-8bd2-73f4171c42a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8055ed7a4d684c96b800c35497b3dbc7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:29.125551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab82d7-05e9-4c14-8bd2-73f4171c42a1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:29.220693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:29.389288 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-52ab82d7-05e9-4c14-8bd2-73f4171c42a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8055ed7a4d684c96b800c35497b3dbc7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:29.592016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-52ab82d7-05e9-4c14-8bd2-73f4171c42a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8055ed7a4d684c96b800c35497b3dbc7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:29.611635 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-52ab82d7-05e9-4c14-8bd2-73f4171c42a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8055ed7a4d684c96b800c35497b3dbc7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:29.619262 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-52ab82d7-05e9-4c14-8bd2-73f4171c42a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8055ed7a4d684c96b800c35497b3dbc7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:29.621449 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-52ab82d7-05e9-4c14-8bd2-73f4171c42a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8055ed7a4d684c96b800c35497b3dbc7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:29.631121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 928/1878] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:29 2022] DELETE /identity/v3/users/8055ed7a4d684c96b800c35497b3dbc7 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:30.301951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:30.303454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:30.318803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5f4a6-f071-4788-81ca-aad6f2368fb5 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:30.440545 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c89e6-c786-4482-bf67-6d36258bf65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:30.441307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c89e6-c786-4482-bf67-6d36258bf65f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:30.441586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c89e6-c786-4482-bf67-6d36258bf65f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:30.441858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c89e6-c786-4482-bf67-6d36258bf65f admin admin] PATH_INFO: `/v3/projects/a4bfda412caa4edc8c3d5112c685e72a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:30.443842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c89e6-c786-4482-bf67-6d36258bf65f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4bfda412caa4edc8c3d5112c685e72a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:30.447005 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c89e6-c786-4482-bf67-6d36258bf65f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:30.610255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:30.657061 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a53c89e6-c786-4482-bf67-6d36258bf65f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4bfda412caa4edc8c3d5112c685e72a', 'request_id': 'req-a53c89e6-c786-4482-bf67-6d36258bf65f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:30.766686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab82d7-05e9-4c14-8bd2-73f4171c42a1 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:30.768406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-52ab82d7-05e9-4c14-8bd2-73f4171c42a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:30.810695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a53c89e6-c786-4482-bf67-6d36258bf65f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4bfda412caa4edc8c3d5112c685e72a', 'request_id': 'req-a53c89e6-c786-4482-bf67-6d36258bf65f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:30.817710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a53c89e6-c786-4482-bf67-6d36258bf65f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4bfda412caa4edc8c3d5112c685e72a', 'request_id': 'req-a53c89e6-c786-4482-bf67-6d36258bf65f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:30.830406 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a53c89e6-c786-4482-bf67-6d36258bf65f admin admin] The token cache is being invalidate because project a4bfda412caa4edc8c3d5112c685e72a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:30.830703 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a53c89e6-c786-4482-bf67-6d36258bf65f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:30.831456 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a53c89e6-c786-4482-bf67-6d36258bf65f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:30.833326 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 951/1879] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:30 2022] DELETE /identity/v3/projects/a4bfda412caa4edc8c3d5112c685e72a => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:30.838064 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d508cd-b374-4943-be2a-e9941357f648 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:30.838417 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d508cd-b374-4943-be2a-e9941357f648 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:30.839349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d508cd-b374-4943-be2a-e9941357f648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:30.840403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab82d7-05e9-4c14-8bd2-73f4171c42a1 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:30.864114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-50d508cd-b374-4943-be2a-e9941357f648 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:30.865556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-50d508cd-b374-4943-be2a-e9941357f648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:30.911661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955c709-488a-45b6-bff7-23ee9a3c237f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:30.912764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955c709-488a-45b6-bff7-23ee9a3c237f service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:30.913144 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955c709-488a-45b6-bff7-23ee9a3c237f service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:30.913517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955c709-488a-45b6-bff7-23ee9a3c237f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:30.915804 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6955c709-488a-45b6-bff7-23ee9a3c237f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:30.947055 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 952/1880] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:30.956119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43049f9-39d5-44d2-b5c4-a877bbd3b529 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:30.957552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43049f9-39d5-44d2-b5c4-a877bbd3b529 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:30.959208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43049f9-39d5-44d2-b5c4-a877bbd3b529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:30.982278 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a43049f9-39d5-44d2-b5c4-a877bbd3b529 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:30.983583 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a43049f9-39d5-44d2-b5c4-a877bbd3b529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:30.992192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 953/1881] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:30.996767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43049f9-39d5-44d2-b5c4-a877bbd3b529 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:31.006658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43049f9-39d5-44d2-b5c4-a877bbd3b529 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:31.016480 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c9cba-78fb-4757-b58d-8443e4cf2546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:31.016970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c9cba-78fb-4757-b58d-8443e4cf2546 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:31.017307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c9cba-78fb-4757-b58d-8443e4cf2546 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:31.017579 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c9cba-78fb-4757-b58d-8443e4cf2546 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:31.018223 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c9cba-78fb-4757-b58d-8443e4cf2546 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:00:31.018520 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c9cba-78fb-4757-b58d-8443e4cf2546 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:31.020285 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c9cba-78fb-4757-b58d-8443e4cf2546 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:31.031029 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955c709-488a-45b6-bff7-23ee9a3c237f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:31.032796 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955c709-488a-45b6-bff7-23ee9a3c237f service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:31.033282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 954/1882] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:00:31.037794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c9cba-78fb-4757-b58d-8443e4cf2546 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:31.045032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 929/1883] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:00:31.047800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c9cba-78fb-4757-b58d-8443e4cf2546 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:31.052481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511da60-e9fa-49b6-bdd2-d16d6e7dd41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:31.053165 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511da60-e9fa-49b6-bdd2-d16d6e7dd41e admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:31.053446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511da60-e9fa-49b6-bdd2-d16d6e7dd41e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:31.053723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511da60-e9fa-49b6-bdd2-d16d6e7dd41e admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:31.054362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511da60-e9fa-49b6-bdd2-d16d6e7dd41e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:31.056304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511da60-e9fa-49b6-bdd2-d16d6e7dd41e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:31.155822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 955/1884] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:31 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:00:31.160027 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955c709-488a-45b6-bff7-23ee9a3c237f service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:31.171362 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955c709-488a-45b6-bff7-23ee9a3c237f service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:31.236578 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c097034-e309-4f9d-9b03-9f75c61be30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:31.237304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c097034-e309-4f9d-9b03-9f75c61be30a admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:31.237585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c097034-e309-4f9d-9b03-9f75c61be30a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:31.237862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c097034-e309-4f9d-9b03-9f75c61be30a admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:31.238498 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c097034-e309-4f9d-9b03-9f75c61be30a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:31.240467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c097034-e309-4f9d-9b03-9f75c61be30a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:31.391816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 930/1885] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:31 2022] POST /identity/v3/users => generated 287 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:31.396383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511da60-e9fa-49b6-bdd2-d16d6e7dd41e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:31.407930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511da60-e9fa-49b6-bdd2-d16d6e7dd41e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:31.412221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464302c3-8815-4151-8110-1384b48dc5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:31.412793 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464302c3-8815-4151-8110-1384b48dc5dd admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:31.413073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464302c3-8815-4151-8110-1384b48dc5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:31.413346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464302c3-8815-4151-8110-1384b48dc5dd admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:31.414272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464302c3-8815-4151-8110-1384b48dc5dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:31.415864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464302c3-8815-4151-8110-1384b48dc5dd admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:31.428171 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 956/1886] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:31.433240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c097034-e309-4f9d-9b03-9f75c61be30a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:31.443806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c097034-e309-4f9d-9b03-9f75c61be30a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:31.448739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760454e-9914-47de-be0d-0b841f7c8a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:31.449412 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760454e-9914-47de-be0d-0b841f7c8a4f admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:31.449687 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760454e-9914-47de-be0d-0b841f7c8a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:31.449955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760454e-9914-47de-be0d-0b841f7c8a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:31.450594 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760454e-9914-47de-be0d-0b841f7c8a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:31.452259 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760454e-9914-47de-be0d-0b841f7c8a4f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:31.462509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 931/1887] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:31.470112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464302c3-8815-4151-8110-1384b48dc5dd admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:31.481351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464302c3-8815-4151-8110-1384b48dc5dd admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:31.486259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e3828-d478-4be8-ba37-60a2bd1bd3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:31.486960 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e3828-d478-4be8-ba37-60a2bd1bd3db admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:31.487244 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e3828-d478-4be8-ba37-60a2bd1bd3db admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:31.487520 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e3828-d478-4be8-ba37-60a2bd1bd3db admin admin] PATH_INFO: `/v3/projects/f36b9db4699345c9a54c39aa803e0984/users/812de7a33ac94119a836b7dd39ea1813/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:31.504754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e3828-d478-4be8-ba37-60a2bd1bd3db admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36b9db4699345c9a54c39aa803e0984, user_id=812de7a33ac94119a836b7dd39ea1813, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:31.507571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e3828-d478-4be8-ba37-60a2bd1bd3db admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:31.621258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 957/1888] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:31 2022] PUT /identity/v3/projects/f36b9db4699345c9a54c39aa803e0984/users/812de7a33ac94119a836b7dd39ea1813/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:31.626592 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9871a97-fec0-4259-8455-2121bae9998e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:31.627138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9871a97-fec0-4259-8455-2121bae9998e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:31.627138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9871a97-fec0-4259-8455-2121bae9998e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:31.646826 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d9871a97-fec0-4259-8455-2121bae9998e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:31.648337 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d9871a97-fec0-4259-8455-2121bae9998e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:31.654239 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:31.728158 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 932/1889] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:31.729427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e3828-d478-4be8-ba37-60a2bd1bd3db admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:31.731904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-408e3828-d478-4be8-ba37-60a2bd1bd3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:31.794028 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e3828-d478-4be8-ba37-60a2bd1bd3db admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:31.798186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f890d-32a3-4680-b048-b33ff5752058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:31.798804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f890d-32a3-4680-b048-b33ff5752058 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:31.799156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f890d-32a3-4680-b048-b33ff5752058 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:31.799618 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f890d-32a3-4680-b048-b33ff5752058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:31.810294 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f890d-32a3-4680-b048-b33ff5752058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:31.811782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f890d-32a3-4680-b048-b33ff5752058 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:31.824478 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 958/1890] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:00:33.646348 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183d13e-b330-4afc-a0e7-b673bebbada6 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:33.648718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183d13e-b330-4afc-a0e7-b673bebbada6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:33.648718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183d13e-b330-4afc-a0e7-b673bebbada6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:33.696786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f183d13e-b330-4afc-a0e7-b673bebbada6 None None] MFA Rules not processed for user `812de7a33ac94119a836b7dd39ea1813`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:33.699513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f183d13e-b330-4afc-a0e7-b673bebbada6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:33.818456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 933/1891] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:33.818871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f890d-32a3-4680-b048-b33ff5752058 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:33.830676 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f890d-32a3-4680-b048-b33ff5752058 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:33.835922 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c236bd9-58d2-4af9-9cdd-ad200a8e2164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:33.836762 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c236bd9-58d2-4af9-9cdd-ad200a8e2164 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:33.837217 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c236bd9-58d2-4af9-9cdd-ad200a8e2164 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:33.837668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c236bd9-58d2-4af9-9cdd-ad200a8e2164 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:33.838307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c236bd9-58d2-4af9-9cdd-ad200a8e2164 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:33.841218 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c236bd9-58d2-4af9-9cdd-ad200a8e2164 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:34.066733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:34.142096 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 959/1892] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:33 2022] POST /identity/v3/projects => generated 373 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:34.142796 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183d13e-b330-4afc-a0e7-b673bebbada6 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:34.154298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c236bd9-58d2-4af9-9cdd-ad200a8e2164 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:34.158251 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6c236bd9-58d2-4af9-9cdd-ad200a8e2164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:34.162290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183d13e-b330-4afc-a0e7-b673bebbada6 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:34.170446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:34.229022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272fb788-f943-4c37-abff-849a87630412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:34.230503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fb788-f943-4c37-abff-849a87630412 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:34.230947 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fb788-f943-4c37-abff-849a87630412 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:34.231255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272fb788-f943-4c37-abff-849a87630412 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:34.231910 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272fb788-f943-4c37-abff-849a87630412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:34.234087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272fb788-f943-4c37-abff-849a87630412 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:34.254248 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c236bd9-58d2-4af9-9cdd-ad200a8e2164 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:34.267712 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439eae0-821f-4794-bd52-b495e5c43830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:34.268465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439eae0-821f-4794-bd52-b495e5c43830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:34.268812 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439eae0-821f-4794-bd52-b495e5c43830 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:34.269202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439eae0-821f-4794-bd52-b495e5c43830 admin admin] PATH_INFO: `/v3/users/c0ba164b85cd4d0899a5f6c7f5b2b6fd` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:34.270494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439eae0-821f-4794-bd52-b495e5c43830 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0ba164b85cd4d0899a5f6c7f5b2b6fd)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:34.272725 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439eae0-821f-4794-bd52-b495e5c43830 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:34.352794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 934/1893] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:34 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:00:34.357284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272fb788-f943-4c37-abff-849a87630412 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:34.368246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272fb788-f943-4c37-abff-849a87630412 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:34.379970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fd276-5ba4-469d-8451-3ef1dc2cb2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:34.380633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fd276-5ba4-469d-8451-3ef1dc2cb2db admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:34.381095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fd276-5ba4-469d-8451-3ef1dc2cb2db admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:34.381513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fd276-5ba4-469d-8451-3ef1dc2cb2db admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:34.383385 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fd276-5ba4-469d-8451-3ef1dc2cb2db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:34.384999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fd276-5ba4-469d-8451-3ef1dc2cb2db admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:34.399633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 935/1894] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:00:34.404809 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fd276-5ba4-469d-8451-3ef1dc2cb2db admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:34.416015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fd276-5ba4-469d-8451-3ef1dc2cb2db admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:34.425667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd848fd-570c-4c57-856a-1a518671edb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:34.426558 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd848fd-570c-4c57-856a-1a518671edb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:34.427011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd848fd-570c-4c57-856a-1a518671edb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:34.427457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd848fd-570c-4c57-856a-1a518671edb9 admin admin] PATH_INFO: `/v3/projects/878db9335f1e48f99bb2d53513aedbf1/users/d6be6f638e7a42628caba6e305d25014/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:34.467167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd848fd-570c-4c57-856a-1a518671edb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=878db9335f1e48f99bb2d53513aedbf1, user_id=d6be6f638e7a42628caba6e305d25014, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:34.467167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd848fd-570c-4c57-856a-1a518671edb9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:34.595258 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 936/1895] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:34 2022] PUT /identity/v3/projects/878db9335f1e48f99bb2d53513aedbf1/users/d6be6f638e7a42628caba6e305d25014/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:34.599098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd848fd-570c-4c57-856a-1a518671edb9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:34.622459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8439eae0-821f-4794-bd52-b495e5c43830 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0ba164b85cd4d0899a5f6c7f5b2b6fd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:34.694458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd848fd-570c-4c57-856a-1a518671edb9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:34.700869 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31b536-1b6a-462b-ba88-03b2f820f5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:34.701640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31b536-1b6a-462b-ba88-03b2f820f5ef admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:34.701922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31b536-1b6a-462b-ba88-03b2f820f5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:34.702197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31b536-1b6a-462b-ba88-03b2f820f5ef admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:34.702842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31b536-1b6a-462b-ba88-03b2f820f5ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:34.704486 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31b536-1b6a-462b-ba88-03b2f820f5ef admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:34.715006 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 937/1896] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:00:34.715970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31b536-1b6a-462b-ba88-03b2f820f5ef admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:34.717696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2c31b536-1b6a-462b-ba88-03b2f820f5ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:34.731260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31b536-1b6a-462b-ba88-03b2f820f5ef admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:34.736182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd34c1a-5cff-43e4-b5b5-d9f5b0197f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:34.736748 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd34c1a-5cff-43e4-b5b5-d9f5b0197f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:34.737031 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd34c1a-5cff-43e4-b5b5-d9f5b0197f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:34.737306 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd34c1a-5cff-43e4-b5b5-d9f5b0197f2b admin admin] PATH_INFO: `/v3/users/fb81ae95248b48c18d5398fea534c8fa` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:34.738443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd34c1a-5cff-43e4-b5b5-d9f5b0197f2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb81ae95248b48c18d5398fea534c8fa)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:34.740442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd34c1a-5cff-43e4-b5b5-d9f5b0197f2b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:34.752001 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8439eae0-821f-4794-bd52-b495e5c43830 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0ba164b85cd4d0899a5f6c7f5b2b6fd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:34.752632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8439eae0-821f-4794-bd52-b495e5c43830 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0ba164b85cd4d0899a5f6c7f5b2b6fd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:34.760463 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8439eae0-821f-4794-bd52-b495e5c43830 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0ba164b85cd4d0899a5f6c7f5b2b6fd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:34.774074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8439eae0-821f-4794-bd52-b495e5c43830 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0ba164b85cd4d0899a5f6c7f5b2b6fd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:34.780273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 960/1897] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:34 2022] DELETE /identity/v3/users/c0ba164b85cd4d0899a5f6c7f5b2b6fd => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:34.781058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439eae0-821f-4794-bd52-b495e5c43830 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:34.782483 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8439eae0-821f-4794-bd52-b495e5c43830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:34.856080 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439eae0-821f-4794-bd52-b495e5c43830 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:34.861069 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e90e6eb-923d-442d-b3b8-f11695159f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:34.861751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90e6eb-923d-442d-b3b8-f11695159f8a admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:34.862037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90e6eb-923d-442d-b3b8-f11695159f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:34.862312 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e90e6eb-923d-442d-b3b8-f11695159f8a admin admin] PATH_INFO: `/v3/projects/878db9335f1e48f99bb2d53513aedbf1/users/d6be6f638e7a42628caba6e305d25014/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:34.864349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90e6eb-923d-442d-b3b8-f11695159f8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=878db9335f1e48f99bb2d53513aedbf1, user_id=d6be6f638e7a42628caba6e305d25014, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:34.867175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e90e6eb-923d-442d-b3b8-f11695159f8a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:34.878939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 961/1898] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:34 2022] PUT /identity/v3/projects/878db9335f1e48f99bb2d53513aedbf1/users/d6be6f638e7a42628caba6e305d25014/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:34.880274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b6a9b-4a9f-4e2c-8395-c12057422434 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:34.880623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b6a9b-4a9f-4e2c-8395-c12057422434 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:34.880959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b6a9b-4a9f-4e2c-8395-c12057422434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:34.903515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-733b6a9b-4a9f-4e2c-8395-c12057422434 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:34.905138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-733b6a9b-4a9f-4e2c-8395-c12057422434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:34.970177 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 962/1899] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:34.987448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b6a9b-4a9f-4e2c-8395-c12057422434 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:34.989598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-733b6a9b-4a9f-4e2c-8395-c12057422434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:35.036282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dbd34c1a-5cff-43e4-b5b5-d9f5b0197f2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb81ae95248b48c18d5398fea534c8fa'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:35.067910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b6a9b-4a9f-4e2c-8395-c12057422434 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:35.072195 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65a4dc-756a-4a24-8052-fd4f9dd289eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:35.072818 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65a4dc-756a-4a24-8052-fd4f9dd289eb service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:35.073146 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65a4dc-756a-4a24-8052-fd4f9dd289eb service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:35.073421 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65a4dc-756a-4a24-8052-fd4f9dd289eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:35.083631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65a4dc-756a-4a24-8052-fd4f9dd289eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:35.085193 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65a4dc-756a-4a24-8052-fd4f9dd289eb service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:35.096078 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 963/1900] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:00:35.203344 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dbd34c1a-5cff-43e4-b5b5-d9f5b0197f2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb81ae95248b48c18d5398fea534c8fa'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:35.221304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dbd34c1a-5cff-43e4-b5b5-d9f5b0197f2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb81ae95248b48c18d5398fea534c8fa'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:35.222463 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dbd34c1a-5cff-43e4-b5b5-d9f5b0197f2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb81ae95248b48c18d5398fea534c8fa'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:35.234639 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dbd34c1a-5cff-43e4-b5b5-d9f5b0197f2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb81ae95248b48c18d5398fea534c8fa'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:35.244349 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 938/1901] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:34 2022] DELETE /identity/v3/users/fb81ae95248b48c18d5398fea534c8fa => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:35.254112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a6f00-ae47-41dd-888c-d22e1e1a079d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:35.254391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a6f00-ae47-41dd-888c-d22e1e1a079d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:35.254682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a6f00-ae47-41dd-888c-d22e1e1a079d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:35.274287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-801a6f00-ae47-41dd-888c-d22e1e1a079d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:35.275784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-801a6f00-ae47-41dd-888c-d22e1e1a079d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:35.362191 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 964/1902] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:35.376003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd34c1a-5cff-43e4-b5b5-d9f5b0197f2b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:35.379869 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-dbd34c1a-5cff-43e4-b5b5-d9f5b0197f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:35.493526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd34c1a-5cff-43e4-b5b5-d9f5b0197f2b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:35.498353 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f06cc-8391-41e2-b6b0-1bb9e6e6a005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:35.499076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f06cc-8391-41e2-b6b0-1bb9e6e6a005 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:35.499448 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f06cc-8391-41e2-b6b0-1bb9e6e6a005 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:35.499930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f06cc-8391-41e2-b6b0-1bb9e6e6a005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:35.513012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f06cc-8391-41e2-b6b0-1bb9e6e6a005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:35.515394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f06cc-8391-41e2-b6b0-1bb9e6e6a005 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:35.534690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 939/1903] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:35.581752 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:35.781924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a6f00-ae47-41dd-888c-d22e1e1a079d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:35.783654 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-801a6f00-ae47-41dd-888c-d22e1e1a079d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:35.809536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a6f00-ae47-41dd-888c-d22e1e1a079d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:35.815294 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ca04c-4191-49ea-aca1-3dc877f69d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:35.816011 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ca04c-4191-49ea-aca1-3dc877f69d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:35.816409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ca04c-4191-49ea-aca1-3dc877f69d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:35.816750 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ca04c-4191-49ea-aca1-3dc877f69d27 admin admin] PATH_INFO: `/v3/projects/53f7dadb6ba6414298e5ebec52c6caa4` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:35.819269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ca04c-4191-49ea-aca1-3dc877f69d27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53f7dadb6ba6414298e5ebec52c6caa4)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:35.821499 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ca04c-4191-49ea-aca1-3dc877f69d27 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:36.036034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-872ca04c-4191-49ea-aca1-3dc877f69d27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53f7dadb6ba6414298e5ebec52c6caa4', 'request_id': 'req-872ca04c-4191-49ea-aca1-3dc877f69d27'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:36.185606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-872ca04c-4191-49ea-aca1-3dc877f69d27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53f7dadb6ba6414298e5ebec52c6caa4', 'request_id': 'req-872ca04c-4191-49ea-aca1-3dc877f69d27'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:36.204010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-872ca04c-4191-49ea-aca1-3dc877f69d27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53f7dadb6ba6414298e5ebec52c6caa4', 'request_id': 'req-872ca04c-4191-49ea-aca1-3dc877f69d27'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:36.222221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-872ca04c-4191-49ea-aca1-3dc877f69d27 admin admin] The token cache is being invalidate because project 53f7dadb6ba6414298e5ebec52c6caa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:36.222911 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-872ca04c-4191-49ea-aca1-3dc877f69d27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:36.224379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-872ca04c-4191-49ea-aca1-3dc877f69d27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:36.226984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 965/1904] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:35 2022] DELETE /identity/v3/projects/53f7dadb6ba6414298e5ebec52c6caa4 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:36.232274 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f06cc-8391-41e2-b6b0-1bb9e6e6a005 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:36.243458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f06cc-8391-41e2-b6b0-1bb9e6e6a005 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:36.348004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baade23-a7a1-4798-a542-0ba779a3a8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:36.349014 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baade23-a7a1-4798-a542-0ba779a3a8cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:36.349297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baade23-a7a1-4798-a542-0ba779a3a8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:36.349570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baade23-a7a1-4798-a542-0ba779a3a8cc admin admin] PATH_INFO: `/v3/users/86111abadd1942bba409ed39ec281aec` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:36.350790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baade23-a7a1-4798-a542-0ba779a3a8cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=86111abadd1942bba409ed39ec281aec)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:36.354383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baade23-a7a1-4798-a542-0ba779a3a8cc admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:36.491852 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:36.659806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4baade23-a7a1-4798-a542-0ba779a3a8cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86111abadd1942bba409ed39ec281aec'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:36.690731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ca04c-4191-49ea-aca1-3dc877f69d27 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:36.692527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-872ca04c-4191-49ea-aca1-3dc877f69d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:36.716949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ca04c-4191-49ea-aca1-3dc877f69d27 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:36.723303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b80d3-7213-4f80-ab18-1802487a6f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:36.724504 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b80d3-7213-4f80-ab18-1802487a6f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:36.737820 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b80d3-7213-4f80-ab18-1802487a6f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:36.738398 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b80d3-7213-4f80-ab18-1802487a6f60 admin admin] PATH_INFO: `/v3/projects/e5d8409e5cc34d1aa04009d96c988496` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:36.740578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7b80d3-7213-4f80-ab18-1802487a6f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d8409e5cc34d1aa04009d96c988496)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:36.743474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7b80d3-7213-4f80-ab18-1802487a6f60 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:36.792438 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4baade23-a7a1-4798-a542-0ba779a3a8cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86111abadd1942bba409ed39ec281aec'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:36.815726 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4baade23-a7a1-4798-a542-0ba779a3a8cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86111abadd1942bba409ed39ec281aec'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:36.817661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4baade23-a7a1-4798-a542-0ba779a3a8cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86111abadd1942bba409ed39ec281aec'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:36.838128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4baade23-a7a1-4798-a542-0ba779a3a8cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86111abadd1942bba409ed39ec281aec'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:36.850550 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 940/1905] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:36 2022] DELETE /identity/v3/users/86111abadd1942bba409ed39ec281aec => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:37.005850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9c7b80d3-7213-4f80-ab18-1802487a6f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d8409e5cc34d1aa04009d96c988496', 'request_id': 'req-9c7b80d3-7213-4f80-ab18-1802487a6f60'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:37.169010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9c7b80d3-7213-4f80-ab18-1802487a6f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d8409e5cc34d1aa04009d96c988496', 'request_id': 'req-9c7b80d3-7213-4f80-ab18-1802487a6f60'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:37.183933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9c7b80d3-7213-4f80-ab18-1802487a6f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d8409e5cc34d1aa04009d96c988496', 'request_id': 'req-9c7b80d3-7213-4f80-ab18-1802487a6f60'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:37.209383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9c7b80d3-7213-4f80-ab18-1802487a6f60 admin admin] The token cache is being invalidate because project e5d8409e5cc34d1aa04009d96c988496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:37.210091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9c7b80d3-7213-4f80-ab18-1802487a6f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:37.211039 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9c7b80d3-7213-4f80-ab18-1802487a6f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:37.213963 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 966/1906] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:36 2022] DELETE /identity/v3/projects/e5d8409e5cc34d1aa04009d96c988496 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:37.227357 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baade23-a7a1-4798-a542-0ba779a3a8cc admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:37.228868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4baade23-a7a1-4798-a542-0ba779a3a8cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:37.337728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baade23-a7a1-4798-a542-0ba779a3a8cc admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:37.414404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f34572-3786-4406-842e-b4419a4dd5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:37.415086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f34572-3786-4406-842e-b4419a4dd5e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:37.415539 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f34572-3786-4406-842e-b4419a4dd5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:37.416104 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f34572-3786-4406-842e-b4419a4dd5e7 admin admin] PATH_INFO: `/v3/users/4374731ce88c4be4ac984f38c2ebb9ab` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:37.417990 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f34572-3786-4406-842e-b4419a4dd5e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4374731ce88c4be4ac984f38c2ebb9ab)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:37.420331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f34572-3786-4406-842e-b4419a4dd5e7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:37.439026 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf34f1-ae81-4f53-a1d1-cac4272f21c1 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:37.439757 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf34f1-ae81-4f53-a1d1-cac4272f21c1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:37.440302 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf34f1-ae81-4f53-a1d1-cac4272f21c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:37.513273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3ecf34f1-ae81-4f53-a1d1-cac4272f21c1 None None] MFA Rules not processed for user `d6be6f638e7a42628caba6e305d25014`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:37.514519 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3ecf34f1-ae81-4f53-a1d1-cac4272f21c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:37.603949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 967/1907] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:37.605246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf34f1-ae81-4f53-a1d1-cac4272f21c1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:37.608281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3ecf34f1-ae81-4f53-a1d1-cac4272f21c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:37.623200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf34f1-ae81-4f53-a1d1-cac4272f21c1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:37.628062 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433bdae1-f283-4d45-9045-41335d244359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:37.628699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bdae1-f283-4d45-9045-41335d244359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:37.629002 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bdae1-f283-4d45-9045-41335d244359 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:37.629271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bdae1-f283-4d45-9045-41335d244359 admin admin] PATH_INFO: `/v3/projects/ac031016944c4da0bb8f09142714b653` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:37.630481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433bdae1-f283-4d45-9045-41335d244359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac031016944c4da0bb8f09142714b653)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:37.632695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433bdae1-f283-4d45-9045-41335d244359 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:37.674660 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-44f34572-3786-4406-842e-b4419a4dd5e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4374731ce88c4be4ac984f38c2ebb9ab'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:37.735115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-44f34572-3786-4406-842e-b4419a4dd5e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4374731ce88c4be4ac984f38c2ebb9ab'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:37.752067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-44f34572-3786-4406-842e-b4419a4dd5e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4374731ce88c4be4ac984f38c2ebb9ab'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:37.752921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-44f34572-3786-4406-842e-b4419a4dd5e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4374731ce88c4be4ac984f38c2ebb9ab'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:37.756415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-433bdae1-f283-4d45-9045-41335d244359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac031016944c4da0bb8f09142714b653', 'request_id': 'req-433bdae1-f283-4d45-9045-41335d244359'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:37.762935 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-44f34572-3786-4406-842e-b4419a4dd5e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4374731ce88c4be4ac984f38c2ebb9ab'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:37.773513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 941/1908] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:37 2022] DELETE /identity/v3/users/4374731ce88c4be4ac984f38c2ebb9ab => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:37.773513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f34572-3786-4406-842e-b4419a4dd5e7 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:37.773513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-44f34572-3786-4406-842e-b4419a4dd5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:37.831920 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-433bdae1-f283-4d45-9045-41335d244359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac031016944c4da0bb8f09142714b653', 'request_id': 'req-433bdae1-f283-4d45-9045-41335d244359'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:37.840000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-433bdae1-f283-4d45-9045-41335d244359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac031016944c4da0bb8f09142714b653', 'request_id': 'req-433bdae1-f283-4d45-9045-41335d244359'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:37.842142 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f34572-3786-4406-842e-b4419a4dd5e7 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:37.847935 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401a916-d045-4286-a71c-79fdbb0cb25f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:37.848214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401a916-d045-4286-a71c-79fdbb0cb25f service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:37.848791 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401a916-d045-4286-a71c-79fdbb0cb25f service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:37.849191 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401a916-d045-4286-a71c-79fdbb0cb25f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:37.851238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c401a916-d045-4286-a71c-79fdbb0cb25f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:37.867959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-433bdae1-f283-4d45-9045-41335d244359 admin admin] The token cache is being invalidate because project ac031016944c4da0bb8f09142714b653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:37.868670 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-433bdae1-f283-4d45-9045-41335d244359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:37.874994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-433bdae1-f283-4d45-9045-41335d244359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:37.880719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 968/1909] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:37 2022] DELETE /identity/v3/projects/ac031016944c4da0bb8f09142714b653 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:37.885762 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c900864-6ce3-4d1a-8047-6d0486132f5d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:37.886083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c900864-6ce3-4d1a-8047-6d0486132f5d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:37.886360 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c900864-6ce3-4d1a-8047-6d0486132f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:37.911856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1c900864-6ce3-4d1a-8047-6d0486132f5d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:37.913106 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1c900864-6ce3-4d1a-8047-6d0486132f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:37.931176 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401a916-d045-4286-a71c-79fdbb0cb25f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:37.933338 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401a916-d045-4286-a71c-79fdbb0cb25f service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:37.945679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 942/1910] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:38.046167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 969/1911] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:38.050838 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9db7e9-b988-4b71-a467-726d5071d6e1 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:38.051901 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9db7e9-b988-4b71-a467-726d5071d6e1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:38.052173 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9db7e9-b988-4b71-a467-726d5071d6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:38.099631 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8f9db7e9-b988-4b71-a467-726d5071d6e1 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:38.101211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8f9db7e9-b988-4b71-a467-726d5071d6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:38.127887 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 943/1912] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:38.131285 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c900864-6ce3-4d1a-8047-6d0486132f5d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:38.145879 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c900864-6ce3-4d1a-8047-6d0486132f5d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:38.156633 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d39bc-1070-4f08-953e-3b5a43fedd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:38.157448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d39bc-1070-4f08-953e-3b5a43fedd69 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:38.157699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d39bc-1070-4f08-953e-3b5a43fedd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:38.157971 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d39bc-1070-4f08-953e-3b5a43fedd69 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:38.158803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d39bc-1070-4f08-953e-3b5a43fedd69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:00:38.159122 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d39bc-1070-4f08-953e-3b5a43fedd69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:38.160745 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d39bc-1070-4f08-953e-3b5a43fedd69 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:38.189546 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 970/1913] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:00:38.194050 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9db7e9-b988-4b71-a467-726d5071d6e1 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:38.208037 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9db7e9-b988-4b71-a467-726d5071d6e1 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:38.222935 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee87eb-f111-4d83-85ea-6a3f8b24b0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:38.223815 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee87eb-f111-4d83-85ea-6a3f8b24b0fe admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:38.224119 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee87eb-f111-4d83-85ea-6a3f8b24b0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:38.224802 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee87eb-f111-4d83-85ea-6a3f8b24b0fe admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:38.225229 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee87eb-f111-4d83-85ea-6a3f8b24b0fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:38.227306 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee87eb-f111-4d83-85ea-6a3f8b24b0fe admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:38.347271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 944/1914] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:38 2022] POST /identity/v3/projects => generated 387 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:38.351685 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d39bc-1070-4f08-953e-3b5a43fedd69 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:38.360322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:38.367522 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d39bc-1070-4f08-953e-3b5a43fedd69 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:38.449614 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f504e-0399-4b57-b653-17bf06e1e54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:38.450320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f504e-0399-4b57-b653-17bf06e1e54e admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:38.450596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f504e-0399-4b57-b653-17bf06e1e54e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:38.450862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f504e-0399-4b57-b653-17bf06e1e54e admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:38.451518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:38.452004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f504e-0399-4b57-b653-17bf06e1e54e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:38.455115 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f504e-0399-4b57-b653-17bf06e1e54e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:38.493339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee87eb-f111-4d83-85ea-6a3f8b24b0fe admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:38.493339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ebee87eb-f111-4d83-85ea-6a3f8b24b0fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:38.517762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee87eb-f111-4d83-85ea-6a3f8b24b0fe admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:38.524677 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065856b0-dbaa-43c6-9039-30990abddec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:38.525688 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065856b0-dbaa-43c6-9039-30990abddec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:38.527027 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065856b0-dbaa-43c6-9039-30990abddec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:38.527695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065856b0-dbaa-43c6-9039-30990abddec3 admin admin] PATH_INFO: `/v3/projects/2a235c37c279428dad3c2fca4a1cc0df` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:38.529258 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065856b0-dbaa-43c6-9039-30990abddec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a235c37c279428dad3c2fca4a1cc0df)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:38.531483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065856b0-dbaa-43c6-9039-30990abddec3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:38.585952 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 971/1915] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:38 2022] POST /identity/v3/users => generated 294 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:38.586951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f504e-0399-4b57-b653-17bf06e1e54e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:38.604877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f504e-0399-4b57-b653-17bf06e1e54e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:38.629255 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa700056-718a-46aa-b916-7cb73b7700a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:38.630185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa700056-718a-46aa-b916-7cb73b7700a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:38.633278 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa700056-718a-46aa-b916-7cb73b7700a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:38.634271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa700056-718a-46aa-b916-7cb73b7700a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:38.636186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa700056-718a-46aa-b916-7cb73b7700a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:38.643566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa700056-718a-46aa-b916-7cb73b7700a3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:38.670305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 972/1916] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:38.675392 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa700056-718a-46aa-b916-7cb73b7700a3 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:38.708246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa700056-718a-46aa-b916-7cb73b7700a3 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:38.746652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-065856b0-dbaa-43c6-9039-30990abddec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a235c37c279428dad3c2fca4a1cc0df', 'request_id': 'req-065856b0-dbaa-43c6-9039-30990abddec3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:38.763328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-065856b0-dbaa-43c6-9039-30990abddec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a235c37c279428dad3c2fca4a1cc0df', 'request_id': 'req-065856b0-dbaa-43c6-9039-30990abddec3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:38.771529 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-065856b0-dbaa-43c6-9039-30990abddec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a235c37c279428dad3c2fca4a1cc0df', 'request_id': 'req-065856b0-dbaa-43c6-9039-30990abddec3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:38.801129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f428bb-de13-4043-bd8f-6e7f403c9c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:38.801835 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f428bb-de13-4043-bd8f-6e7f403c9c8b admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:38.802020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f428bb-de13-4043-bd8f-6e7f403c9c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:38.802276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f428bb-de13-4043-bd8f-6e7f403c9c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:38.802946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f428bb-de13-4043-bd8f-6e7f403c9c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:38.804583 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f428bb-de13-4043-bd8f-6e7f403c9c8b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:38.815465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 973/1917] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:38.820137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f428bb-de13-4043-bd8f-6e7f403c9c8b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:38.830804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f428bb-de13-4043-bd8f-6e7f403c9c8b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:38.836215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba6c7c-67af-4737-adf6-a1187fc92cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:38.836883 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba6c7c-67af-4737-adf6-a1187fc92cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:38.837156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba6c7c-67af-4737-adf6-a1187fc92cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:38.837425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba6c7c-67af-4737-adf6-a1187fc92cc3 admin admin] PATH_INFO: `/v3/projects/9e580f7e23d44c43af8e0829ed8fcf2a/users/90ba836ff6344900bb1d038523811eab/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:38.862122 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba6c7c-67af-4737-adf6-a1187fc92cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e580f7e23d44c43af8e0829ed8fcf2a, user_id=90ba836ff6344900bb1d038523811eab, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:38.864842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba6c7c-67af-4737-adf6-a1187fc92cc3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:38.940163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-065856b0-dbaa-43c6-9039-30990abddec3 admin admin] The token cache is being invalidate because project 2a235c37c279428dad3c2fca4a1cc0df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:38.940163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-065856b0-dbaa-43c6-9039-30990abddec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:38.940163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-065856b0-dbaa-43c6-9039-30990abddec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:38.943776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 945/1918] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:38 2022] DELETE /identity/v3/projects/2a235c37c279428dad3c2fca4a1cc0df => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:38.953942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f4929-276b-4bc8-ad14-5f4fc1988a41 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:38.954419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f4929-276b-4bc8-ad14-5f4fc1988a41 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:38.954693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f4929-276b-4bc8-ad14-5f4fc1988a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:38.972092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 974/1919] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:38 2022] PUT /identity/v3/projects/9e580f7e23d44c43af8e0829ed8fcf2a/users/90ba836ff6344900bb1d038523811eab/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:38.984885 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512277ee-7b62-4392-9419-286e3b0003e7 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:38.984885 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512277ee-7b62-4392-9419-286e3b0003e7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:38.984885 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512277ee-7b62-4392-9419-286e3b0003e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:39.024302 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-3c2f4929-276b-4bc8-ad14-5f4fc1988a41 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:39.025834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-512277ee-7b62-4392-9419-286e3b0003e7 None None] MFA Rules not processed for user `90ba836ff6344900bb1d038523811eab`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:39.027004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3c2f4929-276b-4bc8-ad14-5f4fc1988a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:39.028070 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-512277ee-7b62-4392-9419-286e3b0003e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:39.109476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 975/1920] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:39.114719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512277ee-7b62-4392-9419-286e3b0003e7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:39.115752 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-512277ee-7b62-4392-9419-286e3b0003e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:39.130952 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 946/1921] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:39.139159 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11d136-1c21-46e0-9ece-fb52453f5713 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:39.139529 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11d136-1c21-46e0-9ece-fb52453f5713 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:39.139873 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11d136-1c21-46e0-9ece-fb52453f5713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:39.167600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-bc11d136-1c21-46e0-9ece-fb52453f5713 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:39.168922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bc11d136-1c21-46e0-9ece-fb52453f5713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:39.176980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 947/1922] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:39.186829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11d136-1c21-46e0-9ece-fb52453f5713 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:39.197362 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11d136-1c21-46e0-9ece-fb52453f5713 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:39.202294 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512277ee-7b62-4392-9419-286e3b0003e7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:39.202866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5475c9-58f5-48c9-b4a5-9668d92ed44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:39.203395 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5475c9-58f5-48c9-b4a5-9668d92ed44f admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:39.203635 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5475c9-58f5-48c9-b4a5-9668d92ed44f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:39.203910 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5475c9-58f5-48c9-b4a5-9668d92ed44f admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:39.204563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5475c9-58f5-48c9-b4a5-9668d92ed44f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:00:39.204867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5475c9-58f5-48c9-b4a5-9668d92ed44f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:39.206485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5475c9-58f5-48c9-b4a5-9668d92ed44f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:39.207713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef32f8-003d-4e00-b6f5-76609effd8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:39.208330 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef32f8-003d-4e00-b6f5-76609effd8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:39.208606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef32f8-003d-4e00-b6f5-76609effd8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:39.208873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef32f8-003d-4e00-b6f5-76609effd8e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:39.209507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef32f8-003d-4e00-b6f5-76609effd8e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:39.211450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef32f8-003d-4e00-b6f5-76609effd8e8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:39.220511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 948/1923] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:00:39.227993 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5475c9-58f5-48c9-b4a5-9668d92ed44f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:39.239830 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5475c9-58f5-48c9-b4a5-9668d92ed44f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:39.245294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f80ea-d594-47f7-a101-6c4eb89e4e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:39.245914 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f80ea-d594-47f7-a101-6c4eb89e4e6f admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:39.246362 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f80ea-d594-47f7-a101-6c4eb89e4e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:39.246640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749f80ea-d594-47f7-a101-6c4eb89e4e6f admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:39.247328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f80ea-d594-47f7-a101-6c4eb89e4e6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:39.254168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749f80ea-d594-47f7-a101-6c4eb89e4e6f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:39.283556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:39.330565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 976/1924] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:39 2022] POST /identity/v3/projects => generated 385 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:39.334755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef32f8-003d-4e00-b6f5-76609effd8e8 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:39.367377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 949/1925] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:39 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:00:39.371835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f80ea-d594-47f7-a101-6c4eb89e4e6f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:39.403608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef32f8-003d-4e00-b6f5-76609effd8e8 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:39.472612 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc11a9-aa47-4fb0-b377-d1c55580bc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:39.473542 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc11a9-aa47-4fb0-b377-d1c55580bc3c admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:39.473968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc11a9-aa47-4fb0-b377-d1c55580bc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:39.474388 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc11a9-aa47-4fb0-b377-d1c55580bc3c admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:39.486828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749f80ea-d594-47f7-a101-6c4eb89e4e6f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:39.487758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc11a9-aa47-4fb0-b377-d1c55580bc3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:39.489778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc11a9-aa47-4fb0-b377-d1c55580bc3c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:39.493781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213b1b1-7a18-484f-9bdd-95632f6628d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:39.495939 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213b1b1-7a18-484f-9bdd-95632f6628d0 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:39.496400 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213b1b1-7a18-484f-9bdd-95632f6628d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:39.496945 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213b1b1-7a18-484f-9bdd-95632f6628d0 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:39.497717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213b1b1-7a18-484f-9bdd-95632f6628d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:39.500308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213b1b1-7a18-484f-9bdd-95632f6628d0 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:39.643498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 977/1926] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:39 2022] POST /identity/v3/users => generated 299 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:00:39.647348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc11a9-aa47-4fb0-b377-d1c55580bc3c admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:39.657535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc11a9-aa47-4fb0-b377-d1c55580bc3c admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:39.662361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215a6b2-173f-4b65-bd3a-cba22e97faca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:39.662977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215a6b2-173f-4b65-bd3a-cba22e97faca admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:39.663280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215a6b2-173f-4b65-bd3a-cba22e97faca admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:39.663546 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215a6b2-173f-4b65-bd3a-cba22e97faca admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:39.664210 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215a6b2-173f-4b65-bd3a-cba22e97faca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:39.665801 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215a6b2-173f-4b65-bd3a-cba22e97faca admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:39.679912 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 978/1927] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:00:39.680121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 950/1928] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:39 2022] POST /identity/v3/users => generated 293 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:39.681439 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213b1b1-7a18-484f-9bdd-95632f6628d0 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:39.681574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215a6b2-173f-4b65-bd3a-cba22e97faca admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:39.692226 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213b1b1-7a18-484f-9bdd-95632f6628d0 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:39.692461 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215a6b2-173f-4b65-bd3a-cba22e97faca admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:39.696994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e25bd4-0896-4e24-aa7b-b74a9afe52c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:39.697589 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e25bd4-0896-4e24-aa7b-b74a9afe52c9 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:39.697887 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e25bd4-0896-4e24-aa7b-b74a9afe52c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:39.698067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608de67b-ab05-4000-9766-22642d60e591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:39.698238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e25bd4-0896-4e24-aa7b-b74a9afe52c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:39.698911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e25bd4-0896-4e24-aa7b-b74a9afe52c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:39.699061 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608de67b-ab05-4000-9766-22642d60e591 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:39.699320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608de67b-ab05-4000-9766-22642d60e591 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:39.699597 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608de67b-ab05-4000-9766-22642d60e591 admin admin] PATH_INFO: `/v3/projects/c65682385c8c44f3867f879bd44ab016/users/86995d6556154e438e99073f66071fb9/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:39.700513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e25bd4-0896-4e24-aa7b-b74a9afe52c9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:39.716148 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 951/1929] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:00:39.716714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608de67b-ab05-4000-9766-22642d60e591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c65682385c8c44f3867f879bd44ab016, user_id=86995d6556154e438e99073f66071fb9, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:39.717468 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e25bd4-0896-4e24-aa7b-b74a9afe52c9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:39.719442 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608de67b-ab05-4000-9766-22642d60e591 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:39.727891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e25bd4-0896-4e24-aa7b-b74a9afe52c9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:39.733512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072b6fd-a973-4c64-a13a-e9f7e3e2c93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:39.735011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072b6fd-a973-4c64-a13a-e9f7e3e2c93d admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:39.735295 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072b6fd-a973-4c64-a13a-e9f7e3e2c93d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:39.735574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072b6fd-a973-4c64-a13a-e9f7e3e2c93d admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:39.736233 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072b6fd-a973-4c64-a13a-e9f7e3e2c93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:39.737837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072b6fd-a973-4c64-a13a-e9f7e3e2c93d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:39.750369 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 952/1930] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:39.756131 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072b6fd-a973-4c64-a13a-e9f7e3e2c93d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:39.771505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072b6fd-a973-4c64-a13a-e9f7e3e2c93d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:39.776500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d59bf0-9567-4e4b-a63b-1133972d3e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:39.777163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d59bf0-9567-4e4b-a63b-1133972d3e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:39.777441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d59bf0-9567-4e4b-a63b-1133972d3e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:39.777723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d59bf0-9567-4e4b-a63b-1133972d3e65 admin admin] PATH_INFO: `/v3/projects/5511b49d932748c698eed7f108ff575c/users/d1d58e3303554890954670fd348092fd/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:39.786973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 979/1931] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:39 2022] PUT /identity/v3/projects/c65682385c8c44f3867f879bd44ab016/users/86995d6556154e438e99073f66071fb9/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:39.792863 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608de67b-ab05-4000-9766-22642d60e591 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:39.800914 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d59bf0-9567-4e4b-a63b-1133972d3e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5511b49d932748c698eed7f108ff575c, user_id=d1d58e3303554890954670fd348092fd, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:39.803807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d59bf0-9567-4e4b-a63b-1133972d3e65 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:39.862984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608de67b-ab05-4000-9766-22642d60e591 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:39.867822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be452ef-2e45-4a76-95b3-4839229cf2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:39.868461 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be452ef-2e45-4a76-95b3-4839229cf2ef admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:39.868738 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be452ef-2e45-4a76-95b3-4839229cf2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:39.869009 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be452ef-2e45-4a76-95b3-4839229cf2ef admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:39.869648 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be452ef-2e45-4a76-95b3-4839229cf2ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:39.871848 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be452ef-2e45-4a76-95b3-4839229cf2ef admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:39.882404 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 980/1932] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:39.886574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be452ef-2e45-4a76-95b3-4839229cf2ef admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:39.898884 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be452ef-2e45-4a76-95b3-4839229cf2ef admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:39.902521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 953/1933] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:39 2022] PUT /identity/v3/projects/5511b49d932748c698eed7f108ff575c/users/d1d58e3303554890954670fd348092fd/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:39.922556 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db376814-c932-4dce-be6c-c78c929eae71 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:39.922857 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db376814-c932-4dce-be6c-c78c929eae71 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:39.923160 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db376814-c932-4dce-be6c-c78c929eae71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:39.953785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-db376814-c932-4dce-be6c-c78c929eae71 None None] MFA Rules not processed for user `d1d58e3303554890954670fd348092fd`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:39.955523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-db376814-c932-4dce-be6c-c78c929eae71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:39.965078 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9890ce-5b63-46ee-b918-3c5bf60ddd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:39.966059 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9890ce-5b63-46ee-b918-3c5bf60ddd6b admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:39.966059 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9890ce-5b63-46ee-b918-3c5bf60ddd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:39.966302 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9890ce-5b63-46ee-b918-3c5bf60ddd6b admin admin] PATH_INFO: `/v3/projects/c65682385c8c44f3867f879bd44ab016/users/86995d6556154e438e99073f66071fb9/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:39.968540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9890ce-5b63-46ee-b918-3c5bf60ddd6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c65682385c8c44f3867f879bd44ab016, user_id=86995d6556154e438e99073f66071fb9, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:39.971190 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9890ce-5b63-46ee-b918-3c5bf60ddd6b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:39.983130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 981/1934] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:39 2022] PUT /identity/v3/projects/c65682385c8c44f3867f879bd44ab016/users/86995d6556154e438e99073f66071fb9/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:39.995663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b08eb-d786-4add-a68f-d3f2a7fd843d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:39.995949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b08eb-d786-4add-a68f-d3f2a7fd843d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:39.996268 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b08eb-d786-4add-a68f-d3f2a7fd843d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:40.042397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-167b08eb-d786-4add-a68f-d3f2a7fd843d None None] MFA Rules not processed for user `86995d6556154e438e99073f66071fb9`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:40.042397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-167b08eb-d786-4add-a68f-d3f2a7fd843d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:40.047566 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 954/1935] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:40.052270 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db376814-c932-4dce-be6c-c78c929eae71 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:40.063384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db376814-c932-4dce-be6c-c78c929eae71 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:40.111925 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 982/1936] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:40.123686 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b08eb-d786-4add-a68f-d3f2a7fd843d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:40.125095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-167b08eb-d786-4add-a68f-d3f2a7fd843d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:40.127389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48572403-6e4b-4488-b475-3e2424fb2f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:40.128045 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48572403-6e4b-4488-b475-3e2424fb2f98 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:40.128331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48572403-6e4b-4488-b475-3e2424fb2f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:40.128606 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48572403-6e4b-4488-b475-3e2424fb2f98 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:40.129272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48572403-6e4b-4488-b475-3e2424fb2f98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:40.131236 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48572403-6e4b-4488-b475-3e2424fb2f98 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:40.191301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b08eb-d786-4add-a68f-d3f2a7fd843d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:40.191485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6d319-61bb-429d-8b58-10fb5990e2f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:40.192131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6d319-61bb-429d-8b58-10fb5990e2f0 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:40.192380 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6d319-61bb-429d-8b58-10fb5990e2f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:40.192655 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6d319-61bb-429d-8b58-10fb5990e2f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:40.202857 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee6d319-61bb-429d-8b58-10fb5990e2f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:40.204320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee6d319-61bb-429d-8b58-10fb5990e2f0 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:40.215266 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 983/1937] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:00:40.246877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6d319-61bb-429d-8b58-10fb5990e2f0 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:40.264593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6d319-61bb-429d-8b58-10fb5990e2f0 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:40.272237 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602d51b-97eb-4d5a-971f-2f36fdc67dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:40.276773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602d51b-97eb-4d5a-971f-2f36fdc67dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:40.276773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602d51b-97eb-4d5a-971f-2f36fdc67dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:40.276773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602d51b-97eb-4d5a-971f-2f36fdc67dcb admin admin] PATH_INFO: `/v3/users/90ba836ff6344900bb1d038523811eab` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:40.276773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602d51b-97eb-4d5a-971f-2f36fdc67dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=90ba836ff6344900bb1d038523811eab)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:40.281543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602d51b-97eb-4d5a-971f-2f36fdc67dcb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:40.286938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 955/1938] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:40 2022] POST /identity/v3/projects => generated 383 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:00:40.291158 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48572403-6e4b-4488-b475-3e2424fb2f98 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:40.371421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48572403-6e4b-4488-b475-3e2424fb2f98 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:40.376442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755e5c4-525f-44be-927d-544f2b4d1404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:40.377070 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755e5c4-525f-44be-927d-544f2b4d1404 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:40.377398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755e5c4-525f-44be-927d-544f2b4d1404 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:40.377613 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755e5c4-525f-44be-927d-544f2b4d1404 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:40.378703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e755e5c4-525f-44be-927d-544f2b4d1404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:40.380387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e755e5c4-525f-44be-927d-544f2b4d1404 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:40.527630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 956/1939] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:40 2022] POST /identity/v3/users => generated 298 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:00:40.532758 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755e5c4-525f-44be-927d-544f2b4d1404 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:40.539455 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8602d51b-97eb-4d5a-971f-2f36fdc67dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90ba836ff6344900bb1d038523811eab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:40.543874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755e5c4-525f-44be-927d-544f2b4d1404 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:40.548959 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd26cbd-54c1-4023-a3e2-95a595287166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:40.549575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd26cbd-54c1-4023-a3e2-95a595287166 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:40.549864 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd26cbd-54c1-4023-a3e2-95a595287166 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:40.550120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd26cbd-54c1-4023-a3e2-95a595287166 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:40.550764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd26cbd-54c1-4023-a3e2-95a595287166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:40.552615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd26cbd-54c1-4023-a3e2-95a595287166 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:40.565290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 957/1940] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:40.571021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd26cbd-54c1-4023-a3e2-95a595287166 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:40.581778 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd26cbd-54c1-4023-a3e2-95a595287166 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:40.586751 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e02a43-f6c5-4498-9b1f-1e15507fbf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:40.587470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e02a43-f6c5-4498-9b1f-1e15507fbf54 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:40.587745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e02a43-f6c5-4498-9b1f-1e15507fbf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:40.588019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e02a43-f6c5-4498-9b1f-1e15507fbf54 admin admin] PATH_INFO: `/v3/projects/3811292e03974fe8a376e3ba49d4a72e/users/7240c86492a0498da9a155af963d7de3/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:40.606687 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e02a43-f6c5-4498-9b1f-1e15507fbf54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3811292e03974fe8a376e3ba49d4a72e, user_id=7240c86492a0498da9a155af963d7de3, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:40.609376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e02a43-f6c5-4498-9b1f-1e15507fbf54 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:40.637760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8602d51b-97eb-4d5a-971f-2f36fdc67dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90ba836ff6344900bb1d038523811eab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:40.638442 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8602d51b-97eb-4d5a-971f-2f36fdc67dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90ba836ff6344900bb1d038523811eab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:40.648453 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8602d51b-97eb-4d5a-971f-2f36fdc67dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90ba836ff6344900bb1d038523811eab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:40.664112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8602d51b-97eb-4d5a-971f-2f36fdc67dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90ba836ff6344900bb1d038523811eab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:40.672567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 984/1941] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:40 2022] DELETE /identity/v3/users/90ba836ff6344900bb1d038523811eab => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:40.678162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb1d60-5c53-4548-87da-9952a2c79fa9 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:40.678470 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb1d60-5c53-4548-87da-9952a2c79fa9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:40.678750 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb1d60-5c53-4548-87da-9952a2c79fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:40.709056 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f3eb1d60-5c53-4548-87da-9952a2c79fa9 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:40.711099 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f3eb1d60-5c53-4548-87da-9952a2c79fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:40.728046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 958/1942] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:40 2022] PUT /identity/v3/projects/3811292e03974fe8a376e3ba49d4a72e/users/7240c86492a0498da9a155af963d7de3/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:40.737610 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e02a43-f6c5-4498-9b1f-1e15507fbf54 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:40.742129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f9e02a43-f6c5-4498-9b1f-1e15507fbf54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:40.791356 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 985/1943] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:40.804129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb1d60-5c53-4548-87da-9952a2c79fa9 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:40.805848 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f3eb1d60-5c53-4548-87da-9952a2c79fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:40.839344 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e02a43-f6c5-4498-9b1f-1e15507fbf54 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:40.844037 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:40.846533 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83312ab4-051a-4d95-92a5-a1a54ea6e3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:40.847238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83312ab4-051a-4d95-92a5-a1a54ea6e3ac admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:40.847563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83312ab4-051a-4d95-92a5-a1a54ea6e3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:40.847839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83312ab4-051a-4d95-92a5-a1a54ea6e3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:40.848657 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83312ab4-051a-4d95-92a5-a1a54ea6e3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:40.850465 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83312ab4-051a-4d95-92a5-a1a54ea6e3ac admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:40.862335 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 959/1944] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:40.866314 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83312ab4-051a-4d95-92a5-a1a54ea6e3ac admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:40.873316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb1d60-5c53-4548-87da-9952a2c79fa9 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:40.877555 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83312ab4-051a-4d95-92a5-a1a54ea6e3ac admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:40.884143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bd7a9-2304-4850-be45-9f9d9bcfd2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:40.884508 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bd7a9-2304-4850-be45-9f9d9bcfd2c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:40.884789 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bd7a9-2304-4850-be45-9f9d9bcfd2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:40.885091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bd7a9-2304-4850-be45-9f9d9bcfd2c5 admin admin] PATH_INFO: `/v3/projects/3811292e03974fe8a376e3ba49d4a72e/users/7240c86492a0498da9a155af963d7de3/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:40.887829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bd7a9-2304-4850-be45-9f9d9bcfd2c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3811292e03974fe8a376e3ba49d4a72e, user_id=7240c86492a0498da9a155af963d7de3, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:40.890653 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bd7a9-2304-4850-be45-9f9d9bcfd2c5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:40.901850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 960/1945] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:40 2022] PUT /identity/v3/projects/3811292e03974fe8a376e3ba49d4a72e/users/7240c86492a0498da9a155af963d7de3/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:40.912647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10781adf-f782-48ad-8cbc-9406b7ecc5e8 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:40.912647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10781adf-f782-48ad-8cbc-9406b7ecc5e8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:40.912647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10781adf-f782-48ad-8cbc-9406b7ecc5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:40.924218 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5c46f-a010-4d88-ac7e-2f7641485bf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:40.924831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5c46f-a010-4d88-ac7e-2f7641485bf4 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:40.925115 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5c46f-a010-4d88-ac7e-2f7641485bf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:40.925382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5c46f-a010-4d88-ac7e-2f7641485bf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:40.935903 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5c46f-a010-4d88-ac7e-2f7641485bf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:40.937532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5c46f-a010-4d88-ac7e-2f7641485bf4 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:40.943357 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-10781adf-f782-48ad-8cbc-9406b7ecc5e8 None None] MFA Rules not processed for user `7240c86492a0498da9a155af963d7de3`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:40.944704 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-10781adf-f782-48ad-8cbc-9406b7ecc5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:40.949834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 986/1946] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:41.038715 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 961/1947] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:41.052777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5c46f-a010-4d88-ac7e-2f7641485bf4 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:41.054366 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-aaa5c46f-a010-4d88-ac7e-2f7641485bf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:41.119338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5c46f-a010-4d88-ac7e-2f7641485bf4 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:41.170701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:41.178198 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea742d60-00cb-48c4-b858-b250f8ca2e03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:41.178810 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea742d60-00cb-48c4-b858-b250f8ca2e03 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:41.179129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea742d60-00cb-48c4-b858-b250f8ca2e03 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:41.179404 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea742d60-00cb-48c4-b858-b250f8ca2e03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:41.181145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ea742d60-00cb-48c4-b858-b250f8ca2e03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:41.248918 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea742d60-00cb-48c4-b858-b250f8ca2e03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:41.250370 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea742d60-00cb-48c4-b858-b250f8ca2e03 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:41.260064 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 987/1948] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:41.457530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10781adf-f782-48ad-8cbc-9406b7ecc5e8 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:41.459659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-10781adf-f782-48ad-8cbc-9406b7ecc5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:41.553020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10781adf-f782-48ad-8cbc-9406b7ecc5e8 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:41.557722 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d1850-30ee-4552-8196-1d9c6ac8714d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:41.558350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d1850-30ee-4552-8196-1d9c6ac8714d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:41.558623 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d1850-30ee-4552-8196-1d9c6ac8714d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:41.558915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d1850-30ee-4552-8196-1d9c6ac8714d admin admin] PATH_INFO: `/v3/projects/9e580f7e23d44c43af8e0829ed8fcf2a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:41.560051 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d1850-30ee-4552-8196-1d9c6ac8714d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e580f7e23d44c43af8e0829ed8fcf2a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:41.562025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d1850-30ee-4552-8196-1d9c6ac8714d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:41.757587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd8d1850-30ee-4552-8196-1d9c6ac8714d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e580f7e23d44c43af8e0829ed8fcf2a', 'request_id': 'req-fd8d1850-30ee-4552-8196-1d9c6ac8714d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:41.763588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd8d1850-30ee-4552-8196-1d9c6ac8714d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e580f7e23d44c43af8e0829ed8fcf2a', 'request_id': 'req-fd8d1850-30ee-4552-8196-1d9c6ac8714d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:41.769792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd8d1850-30ee-4552-8196-1d9c6ac8714d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e580f7e23d44c43af8e0829ed8fcf2a', 'request_id': 'req-fd8d1850-30ee-4552-8196-1d9c6ac8714d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:41.846484 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd8d1850-30ee-4552-8196-1d9c6ac8714d admin admin] The token cache is being invalidate because project 9e580f7e23d44c43af8e0829ed8fcf2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:41.846749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd8d1850-30ee-4552-8196-1d9c6ac8714d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:41.847577 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd8d1850-30ee-4552-8196-1d9c6ac8714d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:41.849570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 962/1949] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:41 2022] DELETE /identity/v3/projects/9e580f7e23d44c43af8e0829ed8fcf2a => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:41.853717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea742d60-00cb-48c4-b858-b250f8ca2e03 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:41.873221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea742d60-00cb-48c4-b858-b250f8ca2e03 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:41.951440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6f8f5-a465-4630-a5a6-349bb439e334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:41.952849 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6f8f5-a465-4630-a5a6-349bb439e334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:41.953444 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6f8f5-a465-4630-a5a6-349bb439e334 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:41.954512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6f8f5-a465-4630-a5a6-349bb439e334 admin admin] PATH_INFO: `/v3/users/86995d6556154e438e99073f66071fb9` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:41.956904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6f8f5-a465-4630-a5a6-349bb439e334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86995d6556154e438e99073f66071fb9)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:41.960967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6f8f5-a465-4630-a5a6-349bb439e334 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:42.203510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:42.224529 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-31a6f8f5-a465-4630-a5a6-349bb439e334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86995d6556154e438e99073f66071fb9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:42.369337 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-31a6f8f5-a465-4630-a5a6-349bb439e334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86995d6556154e438e99073f66071fb9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:42.370392 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-31a6f8f5-a465-4630-a5a6-349bb439e334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86995d6556154e438e99073f66071fb9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:42.379293 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-31a6f8f5-a465-4630-a5a6-349bb439e334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86995d6556154e438e99073f66071fb9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:42.394585 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-31a6f8f5-a465-4630-a5a6-349bb439e334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86995d6556154e438e99073f66071fb9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:42.404678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 988/1950] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:41 2022] DELETE /identity/v3/users/86995d6556154e438e99073f66071fb9 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:42.868049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:42.878293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d1850-30ee-4552-8196-1d9c6ac8714d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:42.879919 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fd8d1850-30ee-4552-8196-1d9c6ac8714d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:42.893614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d1850-30ee-4552-8196-1d9c6ac8714d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:42.960836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507d25f-9c7f-48cf-a6e4-a6fc79b58b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:42.961528 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507d25f-9c7f-48cf-a6e4-a6fc79b58b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:42.961831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507d25f-9c7f-48cf-a6e4-a6fc79b58b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:42.962104 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507d25f-9c7f-48cf-a6e4-a6fc79b58b18 admin admin] PATH_INFO: `/v3/projects/c65682385c8c44f3867f879bd44ab016` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:42.964263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507d25f-9c7f-48cf-a6e4-a6fc79b58b18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c65682385c8c44f3867f879bd44ab016)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:42.969383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507d25f-9c7f-48cf-a6e4-a6fc79b58b18 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:43.124319 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0507d25f-9c7f-48cf-a6e4-a6fc79b58b18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c65682385c8c44f3867f879bd44ab016', 'request_id': 'req-0507d25f-9c7f-48cf-a6e4-a6fc79b58b18'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:43.139614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0507d25f-9c7f-48cf-a6e4-a6fc79b58b18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c65682385c8c44f3867f879bd44ab016', 'request_id': 'req-0507d25f-9c7f-48cf-a6e4-a6fc79b58b18'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:43.148233 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0507d25f-9c7f-48cf-a6e4-a6fc79b58b18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c65682385c8c44f3867f879bd44ab016', 'request_id': 'req-0507d25f-9c7f-48cf-a6e4-a6fc79b58b18'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:43.261624 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0507d25f-9c7f-48cf-a6e4-a6fc79b58b18 admin admin] The token cache is being invalidate because project c65682385c8c44f3867f879bd44ab016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:43.264100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0507d25f-9c7f-48cf-a6e4-a6fc79b58b18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:43.264100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0507d25f-9c7f-48cf-a6e4-a6fc79b58b18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:43.264670 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 963/1951] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:42 2022] DELETE /identity/v3/projects/c65682385c8c44f3867f879bd44ab016 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:43.272092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6f8f5-a465-4630-a5a6-349bb439e334 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:43.277763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-31a6f8f5-a465-4630-a5a6-349bb439e334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:43.283118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2fd0d-3cd2-4a93-9c1d-c2cb6f5f9dc3 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:43.283424 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2fd0d-3cd2-4a93-9c1d-c2cb6f5f9dc3 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:43.283684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2fd0d-3cd2-4a93-9c1d-c2cb6f5f9dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:43.309202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-54e2fd0d-3cd2-4a93-9c1d-c2cb6f5f9dc3 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:43.310495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-54e2fd0d-3cd2-4a93-9c1d-c2cb6f5f9dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:43.395154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6f8f5-a465-4630-a5a6-349bb439e334 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:43.431160 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 964/1952] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:43.439852 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd16c2-9f21-4e28-afa6-75a348bfff99 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:43.440190 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd16c2-9f21-4e28-afa6-75a348bfff99 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:43.440600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd16c2-9f21-4e28-afa6-75a348bfff99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:43.487930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-36fd16c2-9f21-4e28-afa6-75a348bfff99 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:43.490438 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-36fd16c2-9f21-4e28-afa6-75a348bfff99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:43.519265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a8a88-a014-4574-8a06-288c7301e5fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:43.520375 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a8a88-a014-4574-8a06-288c7301e5fd service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:43.520375 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a8a88-a014-4574-8a06-288c7301e5fd service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:43.520680 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a8a88-a014-4574-8a06-288c7301e5fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:43.523579 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9f8a8a88-a014-4574-8a06-288c7301e5fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:43.524600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 965/1953] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:43.533621 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd16c2-9f21-4e28-afa6-75a348bfff99 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:43.558069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd16c2-9f21-4e28-afa6-75a348bfff99 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:43.570552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a38adc-7c0e-41f1-9ef1-77b3cb9bc40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:43.573235 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a38adc-7c0e-41f1-9ef1-77b3cb9bc40a admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:43.573645 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a38adc-7c0e-41f1-9ef1-77b3cb9bc40a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:43.574043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a38adc-7c0e-41f1-9ef1-77b3cb9bc40a admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:43.575325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a38adc-7c0e-41f1-9ef1-77b3cb9bc40a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:00:43.575767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a38adc-7c0e-41f1-9ef1-77b3cb9bc40a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:43.581622 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a38adc-7c0e-41f1-9ef1-77b3cb9bc40a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:43.618643 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 966/1954] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:00:43.624375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a38adc-7c0e-41f1-9ef1-77b3cb9bc40a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:43.642693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a8a88-a014-4574-8a06-288c7301e5fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:43.644371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a8a88-a014-4574-8a06-288c7301e5fd service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:43.656925 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a38adc-7c0e-41f1-9ef1-77b3cb9bc40a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:43.671495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cbc5f-a593-4adb-a7ff-b54e40f280cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:43.671495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cbc5f-a593-4adb-a7ff-b54e40f280cd admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:43.671495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cbc5f-a593-4adb-a7ff-b54e40f280cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:43.671495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2cbc5f-a593-4adb-a7ff-b54e40f280cd admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:43.671832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cbc5f-a593-4adb-a7ff-b54e40f280cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:43.671832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2cbc5f-a593-4adb-a7ff-b54e40f280cd admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:43.688878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 989/1955] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:43.868564 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6fd03-50a7-4a8f-b446-4bb72c3b9448 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:43.869805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6fd03-50a7-4a8f-b446-4bb72c3b9448 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:43.870472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6fd03-50a7-4a8f-b446-4bb72c3b9448 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:43.879763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 990/1956] 10.222.0.91 () {54 vars in 921 bytes} [Thu Sep 8 14:00:43 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:00:43.889876 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674f823-3096-42bf-bc5a-0034c9457680 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:43.890483 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674f823-3096-42bf-bc5a-0034c9457680 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:43.890852 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674f823-3096-42bf-bc5a-0034c9457680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:43.919889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 967/1957] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:43 2022] POST /identity/v3/projects => generated 401 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:43.922839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cbc5f-a593-4adb-a7ff-b54e40f280cd admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:43.939637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-2674f823-3096-42bf-bc5a-0034c9457680 None None] MFA Rules not processed for user `38127f41852d412ea6ad270f1618fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:43.942608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2674f823-3096-42bf-bc5a-0034c9457680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:44.021001 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2cbc5f-a593-4adb-a7ff-b54e40f280cd admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:44.027790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb64605-c365-49d1-9677-ef0cb5369fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:44.028406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb64605-c365-49d1-9677-ef0cb5369fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:44.028678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb64605-c365-49d1-9677-ef0cb5369fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:44.028948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb64605-c365-49d1-9677-ef0cb5369fc6 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:44.029585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb64605-c365-49d1-9677-ef0cb5369fc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:44.031545 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb64605-c365-49d1-9677-ef0cb5369fc6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:44.062145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 991/1958] 10.222.0.91 () {58 vars in 1018 bytes} [Thu Sep 8 14:00:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:44.080350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674f823-3096-42bf-bc5a-0034c9457680 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:44.082914 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2674f823-3096-42bf-bc5a-0034c9457680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:44.111484 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674f823-3096-42bf-bc5a-0034c9457680 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:44.121795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad26cf2-c659-4416-a5f7-dae0a113db5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:44.122434 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad26cf2-c659-4416-a5f7-dae0a113db5e service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:44.122930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad26cf2-c659-4416-a5f7-dae0a113db5e service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:44.123254 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad26cf2-c659-4416-a5f7-dae0a113db5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:44.142618 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad26cf2-c659-4416-a5f7-dae0a113db5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:44.142618 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad26cf2-c659-4416-a5f7-dae0a113db5e service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:44.157000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 992/1959] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:44.158207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 968/1960] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:43 2022] POST /identity/v3/users => generated 301 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:00:44.164806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad26cf2-c659-4416-a5f7-dae0a113db5e service nova] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:44.181902 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad26cf2-c659-4416-a5f7-dae0a113db5e service nova] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:44.201052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74152a4-7d66-48c2-afe3-31ec6c635340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:44.214884 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74152a4-7d66-48c2-afe3-31ec6c635340 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:44.217403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74152a4-7d66-48c2-afe3-31ec6c635340 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:44.217585 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74152a4-7d66-48c2-afe3-31ec6c635340 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:44.218252 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74152a4-7d66-48c2-afe3-31ec6c635340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:44.221097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74152a4-7d66-48c2-afe3-31ec6c635340 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:44.243666 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 993/1961] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:44.243778 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb64605-c365-49d1-9677-ef0cb5369fc6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:44.257202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb64605-c365-49d1-9677-ef0cb5369fc6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:44.266758 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cea62c-026c-4ed6-ade1-9e6aa4dabc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:44.267570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cea62c-026c-4ed6-ade1-9e6aa4dabc9e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:44.270641 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cea62c-026c-4ed6-ade1-9e6aa4dabc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:44.270641 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cea62c-026c-4ed6-ade1-9e6aa4dabc9e admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:44.270641 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cea62c-026c-4ed6-ade1-9e6aa4dabc9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:44.270641 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cea62c-026c-4ed6-ade1-9e6aa4dabc9e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:44.305049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 969/1962] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:00:44.307109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74152a4-7d66-48c2-afe3-31ec6c635340 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:44.326328 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74152a4-7d66-48c2-afe3-31ec6c635340 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:44.351496 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:44.363952 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1d47d-b0d6-454b-a526-b27d4bbaed11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:44.364698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1d47d-b0d6-454b-a526-b27d4bbaed11 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:44.365863 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1d47d-b0d6-454b-a526-b27d4bbaed11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:44.375347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1d47d-b0d6-454b-a526-b27d4bbaed11 admin admin] PATH_INFO: `/v3/projects/9befe64a03d54912a9f3f155b66d9d0d/users/cf3c411a658a4420a88743e101778010/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:44.422268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:44.453636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1d47d-b0d6-454b-a526-b27d4bbaed11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9befe64a03d54912a9f3f155b66d9d0d, user_id=cf3c411a658a4420a88743e101778010, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:44.456562 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1d47d-b0d6-454b-a526-b27d4bbaed11 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:44.592263 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 994/1963] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:44 2022] PUT /identity/v3/projects/9befe64a03d54912a9f3f155b66d9d0d/users/cf3c411a658a4420a88743e101778010/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:44.605617 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aba2e4-0fd0-4c60-a571-37eba79e03ef None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:44.606008 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aba2e4-0fd0-4c60-a571-37eba79e03ef None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:44.606306 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aba2e4-0fd0-4c60-a571-37eba79e03ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:44.676584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-89aba2e4-0fd0-4c60-a571-37eba79e03ef None None] MFA Rules not processed for user `cf3c411a658a4420a88743e101778010`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:44.676584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-89aba2e4-0fd0-4c60-a571-37eba79e03ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:44.784325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 970/1964] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:44 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:44.789512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1d47d-b0d6-454b-a526-b27d4bbaed11 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:44.944120 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1d47d-b0d6-454b-a526-b27d4bbaed11 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:44.951377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087de55-5b78-40e2-bde7-100569eba201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:44.952582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087de55-5b78-40e2-bde7-100569eba201 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:44.954249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087de55-5b78-40e2-bde7-100569eba201 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:44.958954 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087de55-5b78-40e2-bde7-100569eba201 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:44.958954 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087de55-5b78-40e2-bde7-100569eba201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:44.958954 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087de55-5b78-40e2-bde7-100569eba201 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:45.211431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 995/1965] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:44 2022] POST /identity/v3/projects => generated 401 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:45.216391 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aba2e4-0fd0-4c60-a571-37eba79e03ef None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:45.239111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aba2e4-0fd0-4c60-a571-37eba79e03ef None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:45.327504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c9ad9-57b0-4ffd-9c05-438a9b85bd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:45.328307 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c9ad9-57b0-4ffd-9c05-438a9b85bd31 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:45.328583 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c9ad9-57b0-4ffd-9c05-438a9b85bd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:45.328854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c9ad9-57b0-4ffd-9c05-438a9b85bd31 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:45.329493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c9ad9-57b0-4ffd-9c05-438a9b85bd31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:45.331493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c9ad9-57b0-4ffd-9c05-438a9b85bd31 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:45.445604 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 971/1966] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:45 2022] POST /identity/v3/users => generated 307 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:00:45.450058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087de55-5b78-40e2-bde7-100569eba201 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:45.461473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087de55-5b78-40e2-bde7-100569eba201 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:45.468472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad8092-67ea-4ccd-8535-0cfd5ed71a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:45.469408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad8092-67ea-4ccd-8535-0cfd5ed71a0b admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:45.469763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad8092-67ea-4ccd-8535-0cfd5ed71a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:45.470159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad8092-67ea-4ccd-8535-0cfd5ed71a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:45.470984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad8092-67ea-4ccd-8535-0cfd5ed71a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:45.472945 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad8092-67ea-4ccd-8535-0cfd5ed71a0b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:45.490034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 996/1967] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:00:45.494250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c9ad9-57b0-4ffd-9c05-438a9b85bd31 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:45.516074 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c9ad9-57b0-4ffd-9c05-438a9b85bd31 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:45.524309 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0984659-eb64-4ae5-8603-45749bd6fc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:45.525538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0984659-eb64-4ae5-8603-45749bd6fc5d admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:45.525769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0984659-eb64-4ae5-8603-45749bd6fc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:45.526692 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0984659-eb64-4ae5-8603-45749bd6fc5d admin admin] PATH_INFO: `/v3/projects/4127b14dff9b43aa83fbf363f5331f61/users/f140b638d3f146d98819e2bd5a97fd61/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:45.550836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0984659-eb64-4ae5-8603-45749bd6fc5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4127b14dff9b43aa83fbf363f5331f61, user_id=f140b638d3f146d98819e2bd5a97fd61, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:45.554753 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0984659-eb64-4ae5-8603-45749bd6fc5d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:45.670807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 972/1968] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:45 2022] PUT /identity/v3/projects/4127b14dff9b43aa83fbf363f5331f61/users/f140b638d3f146d98819e2bd5a97fd61/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:45.674871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad8092-67ea-4ccd-8535-0cfd5ed71a0b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:45.691390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad8092-67ea-4ccd-8535-0cfd5ed71a0b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:45.816475 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c2ebf-2e13-4e7c-ac2b-35777a285411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:45.820697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c2ebf-2e13-4e7c-ac2b-35777a285411 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:45.820697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c2ebf-2e13-4e7c-ac2b-35777a285411 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:45.820697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c2ebf-2e13-4e7c-ac2b-35777a285411 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:45.820697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c2ebf-2e13-4e7c-ac2b-35777a285411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:45.821014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c2ebf-2e13-4e7c-ac2b-35777a285411 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:45.841557 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0984659-eb64-4ae5-8603-45749bd6fc5d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:45.842025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 997/1969] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:45.855784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0984659-eb64-4ae5-8603-45749bd6fc5d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:45.861835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e606abb-e9e6-4710-a852-d08bea9a5203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:45.862582 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e606abb-e9e6-4710-a852-d08bea9a5203 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:45.862852 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e606abb-e9e6-4710-a852-d08bea9a5203 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:45.863586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e606abb-e9e6-4710-a852-d08bea9a5203 admin admin] PATH_INFO: `/v3/projects/4127b14dff9b43aa83fbf363f5331f61/users/f140b638d3f146d98819e2bd5a97fd61/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:45.865435 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e606abb-e9e6-4710-a852-d08bea9a5203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4127b14dff9b43aa83fbf363f5331f61, user_id=f140b638d3f146d98819e2bd5a97fd61, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:45.869902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e606abb-e9e6-4710-a852-d08bea9a5203 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:45.885953 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 973/1970] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:45 2022] PUT /identity/v3/projects/4127b14dff9b43aa83fbf363f5331f61/users/f140b638d3f146d98819e2bd5a97fd61/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:45.900770 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be532a0-1b34-4dda-b67b-8676c037881c None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:45.900770 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be532a0-1b34-4dda-b67b-8676c037881c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:45.901941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be532a0-1b34-4dda-b67b-8676c037881c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:45.934040 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-9be532a0-1b34-4dda-b67b-8676c037881c None None] MFA Rules not processed for user `f140b638d3f146d98819e2bd5a97fd61`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:45.935442 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9be532a0-1b34-4dda-b67b-8676c037881c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:46.023697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 998/1971] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:00:45 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:46.766605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e606abb-e9e6-4710-a852-d08bea9a5203 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:46.773166 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5e606abb-e9e6-4710-a852-d08bea9a5203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:46.849080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e606abb-e9e6-4710-a852-d08bea9a5203 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:46.852921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76de3c-5826-496a-88c9-f3a4836a1835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:46.853969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76de3c-5826-496a-88c9-f3a4836a1835 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:46.854249 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76de3c-5826-496a-88c9-f3a4836a1835 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:46.854521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76de3c-5826-496a-88c9-f3a4836a1835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:46.864438 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76de3c-5826-496a-88c9-f3a4836a1835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:46.865905 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76de3c-5826-496a-88c9-f3a4836a1835 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:46.877293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 974/1972] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:46 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:47.310595 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be532a0-1b34-4dda-b67b-8676c037881c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:47.330350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be532a0-1b34-4dda-b67b-8676c037881c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:47.334622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f939d-0363-4c9d-894c-0255ea1ea120 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:47.335840 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f939d-0363-4c9d-894c-0255ea1ea120 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:47.336339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f939d-0363-4c9d-894c-0255ea1ea120 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:47.336822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f939d-0363-4c9d-894c-0255ea1ea120 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:47.416232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f939d-0363-4c9d-894c-0255ea1ea120 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:47.417846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f939d-0363-4c9d-894c-0255ea1ea120 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:47.429349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 999/1973] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:47 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:47.597432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:47.660523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:49.288938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:51.467954 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76de3c-5826-496a-88c9-f3a4836a1835 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:51.477828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76de3c-5826-496a-88c9-f3a4836a1835 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:51.485390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3dbce-210f-4001-bd48-ea894a3ca785 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:51.486241 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3dbce-210f-4001-bd48-ea894a3ca785 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:51.486631 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3dbce-210f-4001-bd48-ea894a3ca785 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:51.487091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3dbce-210f-4001-bd48-ea894a3ca785 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:51.489457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0bc3dbce-210f-4001-bd48-ea894a3ca785 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:51.555825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3dbce-210f-4001-bd48-ea894a3ca785 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:51.556799 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3dbce-210f-4001-bd48-ea894a3ca785 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:51.567957 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 975/1974] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:51.866674 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:53.684299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:55.094846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:55.860494 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:56.306155 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:57.471776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:57.622646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:59.677946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f939d-0363-4c9d-894c-0255ea1ea120 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:59.757331 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f939d-0363-4c9d-894c-0255ea1ea120 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:59.760338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603cf5c-49e6-4ea2-a29d-ab078cac3912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:59.761611 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603cf5c-49e6-4ea2-a29d-ab078cac3912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:59.761890 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603cf5c-49e6-4ea2-a29d-ab078cac3912 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:59.762160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603cf5c-49e6-4ea2-a29d-ab078cac3912 admin admin] PATH_INFO: `/v3/users/cf3c411a658a4420a88743e101778010` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:59.764027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603cf5c-49e6-4ea2-a29d-ab078cac3912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf3c411a658a4420a88743e101778010)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:59.766215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603cf5c-49e6-4ea2-a29d-ab078cac3912 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:59.969306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:00.063783 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8603cf5c-49e6-4ea2-a29d-ab078cac3912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf3c411a658a4420a88743e101778010'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:00.089198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:00.153493 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8603cf5c-49e6-4ea2-a29d-ab078cac3912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf3c411a658a4420a88743e101778010'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:00.154212 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8603cf5c-49e6-4ea2-a29d-ab078cac3912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf3c411a658a4420a88743e101778010'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:00.162143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8603cf5c-49e6-4ea2-a29d-ab078cac3912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf3c411a658a4420a88743e101778010'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:00.177308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8603cf5c-49e6-4ea2-a29d-ab078cac3912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf3c411a658a4420a88743e101778010'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:00.184071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1000/1975] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:59 2022] DELETE /identity/v3/users/cf3c411a658a4420a88743e101778010 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:00.188577 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6916f-ea2a-4c65-be50-c1c83548d047 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:00.188871 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6916f-ea2a-4c65-be50-c1c83548d047 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:00.189154 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6916f-ea2a-4c65-be50-c1c83548d047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:00.209007 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a9d6916f-ea2a-4c65-be50-c1c83548d047 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:00.210239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a9d6916f-ea2a-4c65-be50-c1c83548d047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:00.283023 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 976/1976] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:00.293572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603cf5c-49e6-4ea2-a29d-ab078cac3912 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:00.295221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8603cf5c-49e6-4ea2-a29d-ab078cac3912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:00.358550 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603cf5c-49e6-4ea2-a29d-ab078cac3912 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:00.365951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a368d2-47cf-491a-b091-81be697ce161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:00.366617 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a368d2-47cf-491a-b091-81be697ce161 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:00.366969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a368d2-47cf-491a-b091-81be697ce161 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:00.367272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a368d2-47cf-491a-b091-81be697ce161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:00.378017 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a368d2-47cf-491a-b091-81be697ce161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:00.385005 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a368d2-47cf-491a-b091-81be697ce161 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:00.390491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1001/1977] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:01.020366 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6916f-ea2a-4c65-be50-c1c83548d047 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:01.022002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a9d6916f-ea2a-4c65-be50-c1c83548d047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:01.054466 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6916f-ea2a-4c65-be50-c1c83548d047 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:01.060492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62116e-9b16-47ee-827d-35e002f71418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:01.061284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62116e-9b16-47ee-827d-35e002f71418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:01.061640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62116e-9b16-47ee-827d-35e002f71418 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:01.062003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62116e-9b16-47ee-827d-35e002f71418 admin admin] PATH_INFO: `/v3/projects/9befe64a03d54912a9f3f155b66d9d0d` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:01.063334 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62116e-9b16-47ee-827d-35e002f71418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9befe64a03d54912a9f3f155b66d9d0d)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:01.065363 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62116e-9b16-47ee-827d-35e002f71418 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:01.217363 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bb62116e-9b16-47ee-827d-35e002f71418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9befe64a03d54912a9f3f155b66d9d0d', 'request_id': 'req-bb62116e-9b16-47ee-827d-35e002f71418'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:01.223121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bb62116e-9b16-47ee-827d-35e002f71418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9befe64a03d54912a9f3f155b66d9d0d', 'request_id': 'req-bb62116e-9b16-47ee-827d-35e002f71418'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:01.229433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bb62116e-9b16-47ee-827d-35e002f71418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9befe64a03d54912a9f3f155b66d9d0d', 'request_id': 'req-bb62116e-9b16-47ee-827d-35e002f71418'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:01.366737 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bb62116e-9b16-47ee-827d-35e002f71418 admin admin] The token cache is being invalidate because project 9befe64a03d54912a9f3f155b66d9d0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:01.367286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bb62116e-9b16-47ee-827d-35e002f71418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:01.368297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bb62116e-9b16-47ee-827d-35e002f71418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:01.370386 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 977/1978] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:01 2022] DELETE /identity/v3/projects/9befe64a03d54912a9f3f155b66d9d0d => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:01.374224 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a368d2-47cf-491a-b091-81be697ce161 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:01.385243 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a368d2-47cf-491a-b091-81be697ce161 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:01.459122 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4162e7-62f6-4cf9-b72e-710efe757901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:01.459759 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4162e7-62f6-4cf9-b72e-710efe757901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:01.460036 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4162e7-62f6-4cf9-b72e-710efe757901 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:01.460310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4162e7-62f6-4cf9-b72e-710efe757901 admin admin] PATH_INFO: `/v3/users/f140b638d3f146d98819e2bd5a97fd61` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:01.461435 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4162e7-62f6-4cf9-b72e-710efe757901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f140b638d3f146d98819e2bd5a97fd61)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:01.463403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4162e7-62f6-4cf9-b72e-710efe757901 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:01.779907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cf4162e7-62f6-4cf9-b72e-710efe757901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f140b638d3f146d98819e2bd5a97fd61'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:01.919381 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cf4162e7-62f6-4cf9-b72e-710efe757901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f140b638d3f146d98819e2bd5a97fd61'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:01.920019 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cf4162e7-62f6-4cf9-b72e-710efe757901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f140b638d3f146d98819e2bd5a97fd61'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:01.935922 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cf4162e7-62f6-4cf9-b72e-710efe757901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f140b638d3f146d98819e2bd5a97fd61'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:01.952638 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cf4162e7-62f6-4cf9-b72e-710efe757901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f140b638d3f146d98819e2bd5a97fd61'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:01.965119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1002/1979] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:01 2022] DELETE /identity/v3/users/f140b638d3f146d98819e2bd5a97fd61 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:02.616469 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62116e-9b16-47ee-827d-35e002f71418 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:02.619715 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bb62116e-9b16-47ee-827d-35e002f71418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:02.631443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62116e-9b16-47ee-827d-35e002f71418 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:02.660851 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:02.693796 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5552198-741f-4f61-84a2-55b716137108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:02.694014 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5552198-741f-4f61-84a2-55b716137108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:02.694292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5552198-741f-4f61-84a2-55b716137108 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:02.694582 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5552198-741f-4f61-84a2-55b716137108 admin admin] PATH_INFO: `/v3/projects/4127b14dff9b43aa83fbf363f5331f61` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:02.695819 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5552198-741f-4f61-84a2-55b716137108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4127b14dff9b43aa83fbf363f5331f61)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:02.697750 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5552198-741f-4f61-84a2-55b716137108 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:02.724517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:02.845800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b5552198-741f-4f61-84a2-55b716137108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4127b14dff9b43aa83fbf363f5331f61', 'request_id': 'req-b5552198-741f-4f61-84a2-55b716137108'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:02.851478 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b5552198-741f-4f61-84a2-55b716137108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4127b14dff9b43aa83fbf363f5331f61', 'request_id': 'req-b5552198-741f-4f61-84a2-55b716137108'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:02.863081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b5552198-741f-4f61-84a2-55b716137108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4127b14dff9b43aa83fbf363f5331f61', 'request_id': 'req-b5552198-741f-4f61-84a2-55b716137108'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:02.957647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b5552198-741f-4f61-84a2-55b716137108 admin admin] The token cache is being invalidate because project 4127b14dff9b43aa83fbf363f5331f61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:02.958429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b5552198-741f-4f61-84a2-55b716137108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:02.958785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b5552198-741f-4f61-84a2-55b716137108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:02.961441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 978/1980] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:02 2022] DELETE /identity/v3/projects/4127b14dff9b43aa83fbf363f5331f61 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:02.967631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef283c26-4d94-4812-b151-b573a83cf1dc None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:02.967743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef283c26-4d94-4812-b151-b573a83cf1dc None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:02.967949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef283c26-4d94-4812-b151-b573a83cf1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:02.989021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ef283c26-4d94-4812-b151-b573a83cf1dc None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:02.990326 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ef283c26-4d94-4812-b151-b573a83cf1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:03.073752 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1003/1981] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:03.079538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04136e3-af37-432f-9c4d-c0331e4aaf03 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:03.079887 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04136e3-af37-432f-9c4d-c0331e4aaf03 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:03.080373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04136e3-af37-432f-9c4d-c0331e4aaf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:03.106924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b04136e3-af37-432f-9c4d-c0331e4aaf03 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:03.108424 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b04136e3-af37-432f-9c4d-c0331e4aaf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:03.118928 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 979/1982] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:03.123731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef283c26-4d94-4812-b151-b573a83cf1dc None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:03.133748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef283c26-4d94-4812-b151-b573a83cf1dc None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:03.139259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cd069-6b87-4340-990e-5042bb028116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:03.139872 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cd069-6b87-4340-990e-5042bb028116 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:03.140153 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cd069-6b87-4340-990e-5042bb028116 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:03.140424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cd069-6b87-4340-990e-5042bb028116 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:03.141076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cd069-6b87-4340-990e-5042bb028116 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:03.141376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cd069-6b87-4340-990e-5042bb028116 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:03.143057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cd069-6b87-4340-990e-5042bb028116 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:03.155594 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1004/1983] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:03.159999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04136e3-af37-432f-9c4d-c0331e4aaf03 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:03.169523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04136e3-af37-432f-9c4d-c0331e4aaf03 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:03.174145 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737743b-0e05-4999-a839-aa68063bc373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:03.174732 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737743b-0e05-4999-a839-aa68063bc373 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:03.175044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737743b-0e05-4999-a839-aa68063bc373 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:03.175312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737743b-0e05-4999-a839-aa68063bc373 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:03.175939 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737743b-0e05-4999-a839-aa68063bc373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:03.177841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737743b-0e05-4999-a839-aa68063bc373 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:03.341900 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 980/1984] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:03 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:03.346408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cd069-6b87-4340-990e-5042bb028116 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:03.356790 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cd069-6b87-4340-990e-5042bb028116 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:03.420485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4724ea-9401-4c68-86c1-5d0d433e5082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:03.421094 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4724ea-9401-4c68-86c1-5d0d433e5082 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:03.421374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4724ea-9401-4c68-86c1-5d0d433e5082 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:03.421634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4724ea-9401-4c68-86c1-5d0d433e5082 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:03.422287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4724ea-9401-4c68-86c1-5d0d433e5082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:03.424589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4724ea-9401-4c68-86c1-5d0d433e5082 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:03.451227 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737743b-0e05-4999-a839-aa68063bc373 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:03.452946 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8737743b-0e05-4999-a839-aa68063bc373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:03.465276 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737743b-0e05-4999-a839-aa68063bc373 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:03.469712 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17d74b-87fd-447f-aee9-e5b3381cf897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:03.470354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17d74b-87fd-447f-aee9-e5b3381cf897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:03.470587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17d74b-87fd-447f-aee9-e5b3381cf897 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:03.470856 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17d74b-87fd-447f-aee9-e5b3381cf897 admin admin] PATH_INFO: `/v3/users/d1d58e3303554890954670fd348092fd` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:03.471994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17d74b-87fd-447f-aee9-e5b3381cf897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d58e3303554890954670fd348092fd)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:03.473907 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17d74b-87fd-447f-aee9-e5b3381cf897 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:03.558428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1005/1985] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:03 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:03.563622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4724ea-9401-4c68-86c1-5d0d433e5082 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:03.574042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4724ea-9401-4c68-86c1-5d0d433e5082 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:03.578643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6b633-62a9-440c-9c49-d56491022df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:03.579444 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6b633-62a9-440c-9c49-d56491022df1 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:03.579721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6b633-62a9-440c-9c49-d56491022df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:03.579992 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6b633-62a9-440c-9c49-d56491022df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:03.580658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6b633-62a9-440c-9c49-d56491022df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:03.586740 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6b633-62a9-440c-9c49-d56491022df1 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:03.593248 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1006/1986] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:03.597160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6b633-62a9-440c-9c49-d56491022df1 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:03.607754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6b633-62a9-440c-9c49-d56491022df1 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:03.612319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b7254-33e5-4d83-9fce-9f8007c420ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:03.612963 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b7254-33e5-4d83-9fce-9f8007c420ab admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:03.613238 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b7254-33e5-4d83-9fce-9f8007c420ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:03.613505 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b7254-33e5-4d83-9fce-9f8007c420ab admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:03.614127 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b7254-33e5-4d83-9fce-9f8007c420ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:03.615768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b7254-33e5-4d83-9fce-9f8007c420ab admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:03.626861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1007/1987] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:03.632819 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b7254-33e5-4d83-9fce-9f8007c420ab admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:03.644378 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b7254-33e5-4d83-9fce-9f8007c420ab admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:03.649353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce25cf5-2af2-426d-bb5c-bab61d97aa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:03.650263 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce25cf5-2af2-426d-bb5c-bab61d97aa69 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:03.650538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce25cf5-2af2-426d-bb5c-bab61d97aa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:03.650804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce25cf5-2af2-426d-bb5c-bab61d97aa69 admin admin] PATH_INFO: `/v3/projects/f62d7c95cfb144009114cf09d5d0319e/users/5f91bc2624a248ec95aec5ca3233a8bc/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:03.667839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce25cf5-2af2-426d-bb5c-bab61d97aa69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f62d7c95cfb144009114cf09d5d0319e, user_id=5f91bc2624a248ec95aec5ca3233a8bc, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:03.670668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce25cf5-2af2-426d-bb5c-bab61d97aa69 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:03.691814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ef17d74b-87fd-447f-aee9-e5b3381cf897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d58e3303554890954670fd348092fd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:03.735858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1008/1988] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:03 2022] PUT /identity/v3/projects/f62d7c95cfb144009114cf09d5d0319e/users/5f91bc2624a248ec95aec5ca3233a8bc/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:03.748200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a0069-5d52-4efd-bd3e-f4ceca0e71aa None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:03.748489 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a0069-5d52-4efd-bd3e-f4ceca0e71aa None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:03.748785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a0069-5d52-4efd-bd3e-f4ceca0e71aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:03.779036 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f85a0069-5d52-4efd-bd3e-f4ceca0e71aa None None] MFA Rules not processed for user `5f91bc2624a248ec95aec5ca3233a8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:03.780268 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f85a0069-5d52-4efd-bd3e-f4ceca0e71aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:03.785928 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ef17d74b-87fd-447f-aee9-e5b3381cf897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d58e3303554890954670fd348092fd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:03.798990 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ef17d74b-87fd-447f-aee9-e5b3381cf897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d58e3303554890954670fd348092fd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:03.799674 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ef17d74b-87fd-447f-aee9-e5b3381cf897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d58e3303554890954670fd348092fd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:03.806449 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ef17d74b-87fd-447f-aee9-e5b3381cf897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d58e3303554890954670fd348092fd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:03.813841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 981/1989] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:03 2022] DELETE /identity/v3/users/d1d58e3303554890954670fd348092fd => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:03.818009 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ed0ff-c3ea-43b7-9455-264e78cbf997 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:03.818523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ed0ff-c3ea-43b7-9455-264e78cbf997 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:03.819179 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ed0ff-c3ea-43b7-9455-264e78cbf997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:03.841855 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-688ed0ff-c3ea-43b7-9455-264e78cbf997 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:03.843503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-688ed0ff-c3ea-43b7-9455-264e78cbf997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:03.854048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1009/1990] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:03.866290 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a0069-5d52-4efd-bd3e-f4ceca0e71aa None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:03.874977 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f85a0069-5d52-4efd-bd3e-f4ceca0e71aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:03.907535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 982/1991] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:03.918299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ed0ff-c3ea-43b7-9455-264e78cbf997 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:03.919802 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-688ed0ff-c3ea-43b7-9455-264e78cbf997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:03.930937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a0069-5d52-4efd-bd3e-f4ceca0e71aa None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:03.984744 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2d1fa-56b4-4915-ade3-2344eae67e40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:03.985369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2d1fa-56b4-4915-ade3-2344eae67e40 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:03.985659 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2d1fa-56b4-4915-ade3-2344eae67e40 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:03.985925 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2d1fa-56b4-4915-ade3-2344eae67e40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:03.988780 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ed0ff-c3ea-43b7-9455-264e78cbf997 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:03.992936 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218fa12-0187-4c6e-ba0d-6f264e78e9d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:03.993690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218fa12-0187-4c6e-ba0d-6f264e78e9d3 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:03.993969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218fa12-0187-4c6e-ba0d-6f264e78e9d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:03.994245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218fa12-0187-4c6e-ba0d-6f264e78e9d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:04.005783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218fa12-0187-4c6e-ba0d-6f264e78e9d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:04.007751 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218fa12-0187-4c6e-ba0d-6f264e78e9d3 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:04.028019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 983/1992] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:01:04.064203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2d1fa-56b4-4915-ade3-2344eae67e40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:04.065748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2d1fa-56b4-4915-ade3-2344eae67e40 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:04.096097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1010/1993] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:01:04.288593 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:04.657295 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218fa12-0187-4c6e-ba0d-6f264e78e9d3 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:04.658765 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f218fa12-0187-4c6e-ba0d-6f264e78e9d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:04.682068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218fa12-0187-4c6e-ba0d-6f264e78e9d3 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:04.690385 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af767eae-da9f-4adb-b0e4-8cf7959fbbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:04.691243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af767eae-da9f-4adb-b0e4-8cf7959fbbde admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:04.691687 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af767eae-da9f-4adb-b0e4-8cf7959fbbde admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:04.692358 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af767eae-da9f-4adb-b0e4-8cf7959fbbde admin admin] PATH_INFO: `/v3/projects/5511b49d932748c698eed7f108ff575c` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:04.694881 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af767eae-da9f-4adb-b0e4-8cf7959fbbde admin admin] RBAC: Authorizing `identity:delete_project(project_id=5511b49d932748c698eed7f108ff575c)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:04.696838 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af767eae-da9f-4adb-b0e4-8cf7959fbbde admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:04.711416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2d1fa-56b4-4915-ade3-2344eae67e40 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:04.713932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-28a2d1fa-56b4-4915-ade3-2344eae67e40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:04.731240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2d1fa-56b4-4915-ade3-2344eae67e40 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:04.742399 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf6e33-cd2e-418d-8648-fab183a19da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:04.743100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf6e33-cd2e-418d-8648-fab183a19da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:04.743409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf6e33-cd2e-418d-8648-fab183a19da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:04.743652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf6e33-cd2e-418d-8648-fab183a19da5 admin admin] PATH_INFO: `/v3/users/d8929f057cc6425b9c9eb3bd2e2f7f62` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:04.744983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf6e33-cd2e-418d-8648-fab183a19da5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8929f057cc6425b9c9eb3bd2e2f7f62)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:04.747360 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf6e33-cd2e-418d-8648-fab183a19da5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:04.849507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:04.867708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-af767eae-da9f-4adb-b0e4-8cf7959fbbde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5511b49d932748c698eed7f108ff575c', 'request_id': 'req-af767eae-da9f-4adb-b0e4-8cf7959fbbde'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:04.873987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-af767eae-da9f-4adb-b0e4-8cf7959fbbde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5511b49d932748c698eed7f108ff575c', 'request_id': 'req-af767eae-da9f-4adb-b0e4-8cf7959fbbde'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:04.880268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-af767eae-da9f-4adb-b0e4-8cf7959fbbde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5511b49d932748c698eed7f108ff575c', 'request_id': 'req-af767eae-da9f-4adb-b0e4-8cf7959fbbde'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:04.980347 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-af767eae-da9f-4adb-b0e4-8cf7959fbbde admin admin] The token cache is being invalidate because project 5511b49d932748c698eed7f108ff575c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:04.980749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-af767eae-da9f-4adb-b0e4-8cf7959fbbde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:04.981589 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-af767eae-da9f-4adb-b0e4-8cf7959fbbde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:04.983330 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 984/1994] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:04 2022] DELETE /identity/v3/projects/5511b49d932748c698eed7f108ff575c => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:04.987252 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af767eae-da9f-4adb-b0e4-8cf7959fbbde admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:04.988654 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-af767eae-da9f-4adb-b0e4-8cf7959fbbde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:04.993455 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4dcf6e33-cd2e-418d-8648-fab183a19da5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8929f057cc6425b9c9eb3bd2e2f7f62'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.058772 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af767eae-da9f-4adb-b0e4-8cf7959fbbde admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:05.063940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474f1df-4592-41db-a40b-cc11e0e9bf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:05.064827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474f1df-4592-41db-a40b-cc11e0e9bf02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:05.065158 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474f1df-4592-41db-a40b-cc11e0e9bf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:05.065430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474f1df-4592-41db-a40b-cc11e0e9bf02 admin admin] PATH_INFO: `/v3/users/7240c86492a0498da9a155af963d7de3` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:05.067052 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474f1df-4592-41db-a40b-cc11e0e9bf02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7240c86492a0498da9a155af963d7de3)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:05.069033 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474f1df-4592-41db-a40b-cc11e0e9bf02 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:05.089017 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4dcf6e33-cd2e-418d-8648-fab183a19da5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8929f057cc6425b9c9eb3bd2e2f7f62'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.089845 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4dcf6e33-cd2e-418d-8648-fab183a19da5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8929f057cc6425b9c9eb3bd2e2f7f62'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.097561 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4dcf6e33-cd2e-418d-8648-fab183a19da5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8929f057cc6425b9c9eb3bd2e2f7f62'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.110847 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4dcf6e33-cd2e-418d-8648-fab183a19da5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8929f057cc6425b9c9eb3bd2e2f7f62'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.117138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1011/1995] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:04 2022] DELETE /identity/v3/users/d8929f057cc6425b9c9eb3bd2e2f7f62 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:05.122105 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ff2b8-d131-441a-86f2-172f1bb33510 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:05.122445 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ff2b8-d131-441a-86f2-172f1bb33510 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:05.122727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ff2b8-d131-441a-86f2-172f1bb33510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:05.142200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c83ff2b8-d131-441a-86f2-172f1bb33510 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:05.143438 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c83ff2b8-d131-441a-86f2-172f1bb33510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:05.209352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1012/1996] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:05.219186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ff2b8-d131-441a-86f2-172f1bb33510 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:05.220910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c83ff2b8-d131-441a-86f2-172f1bb33510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:05.311190 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ff2b8-d131-441a-86f2-172f1bb33510 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:05.314933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918112b-aa5f-4c83-9b8e-9cb671b2710d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:05.315546 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918112b-aa5f-4c83-9b8e-9cb671b2710d service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:05.315859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918112b-aa5f-4c83-9b8e-9cb671b2710d service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:05.316130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918112b-aa5f-4c83-9b8e-9cb671b2710d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:05.325534 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918112b-aa5f-4c83-9b8e-9cb671b2710d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:05.326976 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918112b-aa5f-4c83-9b8e-9cb671b2710d service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:05.336438 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1013/1997] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:05.427652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3474f1df-4592-41db-a40b-cc11e0e9bf02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7240c86492a0498da9a155af963d7de3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.557497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3474f1df-4592-41db-a40b-cc11e0e9bf02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7240c86492a0498da9a155af963d7de3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.571274 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3474f1df-4592-41db-a40b-cc11e0e9bf02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7240c86492a0498da9a155af963d7de3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.571995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3474f1df-4592-41db-a40b-cc11e0e9bf02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7240c86492a0498da9a155af963d7de3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.579410 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3474f1df-4592-41db-a40b-cc11e0e9bf02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7240c86492a0498da9a155af963d7de3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.585754 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 985/1998] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:04 2022] DELETE /identity/v3/users/7240c86492a0498da9a155af963d7de3 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:05.734771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:05.789810 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918112b-aa5f-4c83-9b8e-9cb671b2710d service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:05.790359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2918112b-aa5f-4c83-9b8e-9cb671b2710d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:05.815753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918112b-aa5f-4c83-9b8e-9cb671b2710d service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:05.921138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed8880-94ca-4966-b3bd-2470b5380d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:05.921802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed8880-94ca-4966-b3bd-2470b5380d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:05.922091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed8880-94ca-4966-b3bd-2470b5380d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:05.922383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed8880-94ca-4966-b3bd-2470b5380d98 admin admin] PATH_INFO: `/v3/projects/f3d2e282bb3948b281fb311316e5172a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:05.927374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed8880-94ca-4966-b3bd-2470b5380d98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3d2e282bb3948b281fb311316e5172a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:05.929370 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed8880-94ca-4966-b3bd-2470b5380d98 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:06.082003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474f1df-4592-41db-a40b-cc11e0e9bf02 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:06.083612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3474f1df-4592-41db-a40b-cc11e0e9bf02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:06.099515 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474f1df-4592-41db-a40b-cc11e0e9bf02 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:06.115927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5eed8880-94ca-4966-b3bd-2470b5380d98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3d2e282bb3948b281fb311316e5172a', 'request_id': 'req-5eed8880-94ca-4966-b3bd-2470b5380d98'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:06.163041 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a19c5-5198-4351-b39c-28b0b86f1c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:06.163716 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a19c5-5198-4351-b39c-28b0b86f1c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.163997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a19c5-5198-4351-b39c-28b0b86f1c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.164268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a19c5-5198-4351-b39c-28b0b86f1c64 admin admin] PATH_INFO: `/v3/projects/3811292e03974fe8a376e3ba49d4a72e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.165446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a19c5-5198-4351-b39c-28b0b86f1c64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3811292e03974fe8a376e3ba49d4a72e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:06.168041 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a19c5-5198-4351-b39c-28b0b86f1c64 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:06.278110 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5eed8880-94ca-4966-b3bd-2470b5380d98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3d2e282bb3948b281fb311316e5172a', 'request_id': 'req-5eed8880-94ca-4966-b3bd-2470b5380d98'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:06.286677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5eed8880-94ca-4966-b3bd-2470b5380d98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3d2e282bb3948b281fb311316e5172a', 'request_id': 'req-5eed8880-94ca-4966-b3bd-2470b5380d98'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:06.298392 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5eed8880-94ca-4966-b3bd-2470b5380d98 admin admin] The token cache is being invalidate because project f3d2e282bb3948b281fb311316e5172a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:06.298679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5eed8880-94ca-4966-b3bd-2470b5380d98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:06.299371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5eed8880-94ca-4966-b3bd-2470b5380d98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:06.300891 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1014/1999] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:05 2022] DELETE /identity/v3/projects/f3d2e282bb3948b281fb311316e5172a => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:06.304901 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed8880-94ca-4966-b3bd-2470b5380d98 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:06.306214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5eed8880-94ca-4966-b3bd-2470b5380d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:06.318576 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed8880-94ca-4966-b3bd-2470b5380d98 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:06.327203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae63df0-aa80-4959-9124-bcbb737d2467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:06.336277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae63df0-aa80-4959-9124-bcbb737d2467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.336277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae63df0-aa80-4959-9124-bcbb737d2467 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.336277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae63df0-aa80-4959-9124-bcbb737d2467 admin admin] PATH_INFO: `/v3/users/e8f60f42606b4c49a6a9194a1545f9cb` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.336277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae63df0-aa80-4959-9124-bcbb737d2467 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8f60f42606b4c49a6a9194a1545f9cb)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:06.336277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae63df0-aa80-4959-9124-bcbb737d2467 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:06.352290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-800a19c5-5198-4351-b39c-28b0b86f1c64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3811292e03974fe8a376e3ba49d4a72e', 'request_id': 'req-800a19c5-5198-4351-b39c-28b0b86f1c64'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:06.362054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-800a19c5-5198-4351-b39c-28b0b86f1c64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3811292e03974fe8a376e3ba49d4a72e', 'request_id': 'req-800a19c5-5198-4351-b39c-28b0b86f1c64'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:06.369397 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-800a19c5-5198-4351-b39c-28b0b86f1c64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3811292e03974fe8a376e3ba49d4a72e', 'request_id': 'req-800a19c5-5198-4351-b39c-28b0b86f1c64'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:06.448674 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-800a19c5-5198-4351-b39c-28b0b86f1c64 admin admin] The token cache is being invalidate because project 3811292e03974fe8a376e3ba49d4a72e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:06.449068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-800a19c5-5198-4351-b39c-28b0b86f1c64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:06.449712 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-800a19c5-5198-4351-b39c-28b0b86f1c64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:06.451262 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 986/2000] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:06 2022] DELETE /identity/v3/projects/3811292e03974fe8a376e3ba49d4a72e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:06.457730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf95cb1-ca22-4abe-81dc-08d12689a4b4 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.458130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf95cb1-ca22-4abe-81dc-08d12689a4b4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.459157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf95cb1-ca22-4abe-81dc-08d12689a4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.481841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7bf95cb1-ca22-4abe-81dc-08d12689a4b4 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:06.483141 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7bf95cb1-ca22-4abe-81dc-08d12689a4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:06.550867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 987/2001] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:06.559887 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48817f45-ffee-4aa4-b2c1-dc1025dec52a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.560124 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48817f45-ffee-4aa4-b2c1-dc1025dec52a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.560398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48817f45-ffee-4aa4-b2c1-dc1025dec52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.564988 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4ae63df0-aa80-4959-9124-bcbb737d2467 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8f60f42606b4c49a6a9194a1545f9cb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:06.580783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-48817f45-ffee-4aa4-b2c1-dc1025dec52a None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:06.582002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-48817f45-ffee-4aa4-b2c1-dc1025dec52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:06.590749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 988/2002] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:06.595023 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48817f45-ffee-4aa4-b2c1-dc1025dec52a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:06.604373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48817f45-ffee-4aa4-b2c1-dc1025dec52a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:06.609146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d903b-60e1-43fc-9b66-6f64e11630f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:06.609733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d903b-60e1-43fc-9b66-6f64e11630f8 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.610019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d903b-60e1-43fc-9b66-6f64e11630f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.610284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d903b-60e1-43fc-9b66-6f64e11630f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.611002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d903b-60e1-43fc-9b66-6f64e11630f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:06.615608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d903b-60e1-43fc-9b66-6f64e11630f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:06.615608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d903b-60e1-43fc-9b66-6f64e11630f8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:06.625703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 989/2003] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:01:06.630262 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d903b-60e1-43fc-9b66-6f64e11630f8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:06.644852 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d903b-60e1-43fc-9b66-6f64e11630f8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:06.656292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2678964-f634-4ad3-95a4-8dc9d0909b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:06.656918 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2678964-f634-4ad3-95a4-8dc9d0909b9d admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.657199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2678964-f634-4ad3-95a4-8dc9d0909b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.657473 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2678964-f634-4ad3-95a4-8dc9d0909b9d admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.658137 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2678964-f634-4ad3-95a4-8dc9d0909b9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:06.660073 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2678964-f634-4ad3-95a4-8dc9d0909b9d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:06.673111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4ae63df0-aa80-4959-9124-bcbb737d2467 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8f60f42606b4c49a6a9194a1545f9cb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:06.673111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4ae63df0-aa80-4959-9124-bcbb737d2467 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8f60f42606b4c49a6a9194a1545f9cb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:06.693848 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4ae63df0-aa80-4959-9124-bcbb737d2467 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8f60f42606b4c49a6a9194a1545f9cb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:06.718773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4ae63df0-aa80-4959-9124-bcbb737d2467 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8f60f42606b4c49a6a9194a1545f9cb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:06.727262 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1015/2004] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:06 2022] DELETE /identity/v3/users/e8f60f42606b4c49a6a9194a1545f9cb => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:06.785071 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 990/2005] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:06 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:06.789262 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae63df0-aa80-4959-9124-bcbb737d2467 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:06.790264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4ae63df0-aa80-4959-9124-bcbb737d2467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:06.838762 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:06.861950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae63df0-aa80-4959-9124-bcbb737d2467 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:06.921261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e86985-e399-42c9-80b1-4b9d5b95530d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:06.921949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e86985-e399-42c9-80b1-4b9d5b95530d admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.922233 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e86985-e399-42c9-80b1-4b9d5b95530d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.922509 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e86985-e399-42c9-80b1-4b9d5b95530d admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.923225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e86985-e399-42c9-80b1-4b9d5b95530d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:06.925166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e86985-e399-42c9-80b1-4b9d5b95530d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:07.015209 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1016/2006] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:06 2022] POST /identity/v3/users => generated 287 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:07.019821 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2678964-f634-4ad3-95a4-8dc9d0909b9d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:07.031545 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2678964-f634-4ad3-95a4-8dc9d0909b9d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:07.036463 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e1c77-6a71-473e-a5e4-6d8006017752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:07.037234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e1c77-6a71-473e-a5e4-6d8006017752 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.037544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e1c77-6a71-473e-a5e4-6d8006017752 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.037816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e1c77-6a71-473e-a5e4-6d8006017752 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.038474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e1c77-6a71-473e-a5e4-6d8006017752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:07.040149 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e1c77-6a71-473e-a5e4-6d8006017752 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:07.057499 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 991/2007] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:07.058657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e86985-e399-42c9-80b1-4b9d5b95530d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:07.070620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e86985-e399-42c9-80b1-4b9d5b95530d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:07.076072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b71cbb-f2a3-4e59-9d4b-22ac75b6063a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:07.076673 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b71cbb-f2a3-4e59-9d4b-22ac75b6063a admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.076946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b71cbb-f2a3-4e59-9d4b-22ac75b6063a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.077225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b71cbb-f2a3-4e59-9d4b-22ac75b6063a admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.077898 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b71cbb-f2a3-4e59-9d4b-22ac75b6063a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:07.079540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b71cbb-f2a3-4e59-9d4b-22ac75b6063a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:07.090849 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1017/2008] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:07.095443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e1c77-6a71-473e-a5e4-6d8006017752 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:07.106578 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e1c77-6a71-473e-a5e4-6d8006017752 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:07.111613 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb939c-0266-4249-9cf1-b4f312f5a872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:07.112297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb939c-0266-4249-9cf1-b4f312f5a872 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.112574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb939c-0266-4249-9cf1-b4f312f5a872 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.112870 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb939c-0266-4249-9cf1-b4f312f5a872 admin admin] PATH_INFO: `/v3/projects/b29ed660360d4164a58e4d2a1663c51b/users/c5a8dd36baa84ab7a0778f20c8502ce7/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.127949 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb939c-0266-4249-9cf1-b4f312f5a872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b29ed660360d4164a58e4d2a1663c51b, user_id=c5a8dd36baa84ab7a0778f20c8502ce7, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:07.130616 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb939c-0266-4249-9cf1-b4f312f5a872 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:07.177221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b71cbb-f2a3-4e59-9d4b-22ac75b6063a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:07.179072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c2b71cbb-f2a3-4e59-9d4b-22ac75b6063a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:07.194116 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b71cbb-f2a3-4e59-9d4b-22ac75b6063a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:07.205672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7601d65-e20b-432c-8386-ec13cc4c0e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:07.206596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7601d65-e20b-432c-8386-ec13cc4c0e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.207047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7601d65-e20b-432c-8386-ec13cc4c0e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.207703 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7601d65-e20b-432c-8386-ec13cc4c0e90 admin admin] PATH_INFO: `/v3/projects/20b10de4152748169f1e7d3246d2a883` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.208528 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 992/2009] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:07 2022] PUT /identity/v3/projects/b29ed660360d4164a58e4d2a1663c51b/users/c5a8dd36baa84ab7a0778f20c8502ce7/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:07.210130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7601d65-e20b-432c-8386-ec13cc4c0e90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20b10de4152748169f1e7d3246d2a883)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:07.214337 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7601d65-e20b-432c-8386-ec13cc4c0e90 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:07.229156 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574e32e-442c-4fa4-b82b-2986c5de50e4 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.229473 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574e32e-442c-4fa4-b82b-2986c5de50e4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.230346 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574e32e-442c-4fa4-b82b-2986c5de50e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.259099 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e574e32e-442c-4fa4-b82b-2986c5de50e4 None None] MFA Rules not processed for user `c5a8dd36baa84ab7a0778f20c8502ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:07.260485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e574e32e-442c-4fa4-b82b-2986c5de50e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:07.324702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 993/2010] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:07.336839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574e32e-442c-4fa4-b82b-2986c5de50e4 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:07.338230 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e574e32e-442c-4fa4-b82b-2986c5de50e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:07.344226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b7601d65-e20b-432c-8386-ec13cc4c0e90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20b10de4152748169f1e7d3246d2a883', 'request_id': 'req-b7601d65-e20b-432c-8386-ec13cc4c0e90'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:07.398352 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574e32e-442c-4fa4-b82b-2986c5de50e4 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:07.403297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b7601d65-e20b-432c-8386-ec13cc4c0e90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20b10de4152748169f1e7d3246d2a883', 'request_id': 'req-b7601d65-e20b-432c-8386-ec13cc4c0e90'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:07.410710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b7601d65-e20b-432c-8386-ec13cc4c0e90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20b10de4152748169f1e7d3246d2a883', 'request_id': 'req-b7601d65-e20b-432c-8386-ec13cc4c0e90'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:07.424555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b7601d65-e20b-432c-8386-ec13cc4c0e90 admin admin] The token cache is being invalidate because project 20b10de4152748169f1e7d3246d2a883 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:07.425315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b7601d65-e20b-432c-8386-ec13cc4c0e90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:07.426000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b7601d65-e20b-432c-8386-ec13cc4c0e90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:07.427602 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1018/2011] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:07 2022] DELETE /identity/v3/projects/20b10de4152748169f1e7d3246d2a883 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:07.432612 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa376d-9099-4214-b3c3-882e8fe0c2b6 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.433656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa376d-9099-4214-b3c3-882e8fe0c2b6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.433656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa376d-9099-4214-b3c3-882e8fe0c2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.449700 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1e069-50c6-4717-86ba-ec4f1bfa7091 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:07.450326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1e069-50c6-4717-86ba-ec4f1bfa7091 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.450607 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1e069-50c6-4717-86ba-ec4f1bfa7091 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.450905 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1e069-50c6-4717-86ba-ec4f1bfa7091 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.452680 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-eba1e069-50c6-4717-86ba-ec4f1bfa7091 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:07.454127 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-80fa376d-9099-4214-b3c3-882e8fe0c2b6 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:07.455334 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-80fa376d-9099-4214-b3c3-882e8fe0c2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:07.529622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1019/2012] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:07.530344 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:07.530344 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1e069-50c6-4717-86ba-ec4f1bfa7091 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:07.530344 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1e069-50c6-4717-86ba-ec4f1bfa7091 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:07.531474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aa8cc-b723-4d69-a9dc-fc20a7d534ac None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.531819 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aa8cc-b723-4d69-a9dc-fc20a7d534ac None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.532158 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aa8cc-b723-4d69-a9dc-fc20a7d534ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.539797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 994/2013] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:01:07.562809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-882aa8cc-b723-4d69-a9dc-fc20a7d534ac None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:07.562809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-882aa8cc-b723-4d69-a9dc-fc20a7d534ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:07.564052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1020/2014] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:07.564363 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1e069-50c6-4717-86ba-ec4f1bfa7091 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:07.565898 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-eba1e069-50c6-4717-86ba-ec4f1bfa7091 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:07.567860 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aa8cc-b723-4d69-a9dc-fc20a7d534ac None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:07.579239 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aa8cc-b723-4d69-a9dc-fc20a7d534ac None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:07.582410 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1e069-50c6-4717-86ba-ec4f1bfa7091 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:07.582825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4836d51-1c59-4ebd-a69e-89db76e7a07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:07.583573 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4836d51-1c59-4ebd-a69e-89db76e7a07d admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.583969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4836d51-1c59-4ebd-a69e-89db76e7a07d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.584308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4836d51-1c59-4ebd-a69e-89db76e7a07d admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.585044 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4836d51-1c59-4ebd-a69e-89db76e7a07d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:07.585419 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4836d51-1c59-4ebd-a69e-89db76e7a07d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:07.587121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4836d51-1c59-4ebd-a69e-89db76e7a07d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:07.587824 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8d335-ccb3-4446-92d2-1704505cde44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:07.588751 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8d335-ccb3-4446-92d2-1704505cde44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.589540 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8d335-ccb3-4446-92d2-1704505cde44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.589921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8d335-ccb3-4446-92d2-1704505cde44 admin admin] PATH_INFO: `/v3/users/c5a8dd36baa84ab7a0778f20c8502ce7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.591210 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8d335-ccb3-4446-92d2-1704505cde44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5a8dd36baa84ab7a0778f20c8502ce7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:07.593223 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8d335-ccb3-4446-92d2-1704505cde44 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:07.603771 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1021/2015] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:01:07.607954 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4836d51-1c59-4ebd-a69e-89db76e7a07d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:07.621349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4836d51-1c59-4ebd-a69e-89db76e7a07d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:07.627496 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f330e-6ed2-4ce6-8995-02c775e802ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:07.628183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f330e-6ed2-4ce6-8995-02c775e802ea admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.628444 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f330e-6ed2-4ce6-8995-02c775e802ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.628748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f330e-6ed2-4ce6-8995-02c775e802ea admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.629428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f330e-6ed2-4ce6-8995-02c775e802ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:07.631588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f330e-6ed2-4ce6-8995-02c775e802ea admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:07.794164 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1022/2016] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:07 2022] POST /identity/v3/projects => generated 365 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:07.796156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f330e-6ed2-4ce6-8995-02c775e802ea admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:07.800358 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e2c8d335-ccb3-4446-92d2-1704505cde44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5a8dd36baa84ab7a0778f20c8502ce7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:07.859862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f330e-6ed2-4ce6-8995-02c775e802ea admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:07.864632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73e3c1-b534-492c-bc8f-beabf307abc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:07.869389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73e3c1-b534-492c-bc8f-beabf307abc3 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.869389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73e3c1-b534-492c-bc8f-beabf307abc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.869389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a73e3c1-b534-492c-bc8f-beabf307abc3 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.869389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73e3c1-b534-492c-bc8f-beabf307abc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:07.869389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a73e3c1-b534-492c-bc8f-beabf307abc3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:07.895243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e2c8d335-ccb3-4446-92d2-1704505cde44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5a8dd36baa84ab7a0778f20c8502ce7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:07.909321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e2c8d335-ccb3-4446-92d2-1704505cde44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5a8dd36baa84ab7a0778f20c8502ce7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:07.910061 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e2c8d335-ccb3-4446-92d2-1704505cde44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5a8dd36baa84ab7a0778f20c8502ce7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:07.917877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e2c8d335-ccb3-4446-92d2-1704505cde44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5a8dd36baa84ab7a0778f20c8502ce7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:07.924486 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 995/2017] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:07 2022] DELETE /identity/v3/users/c5a8dd36baa84ab7a0778f20c8502ce7 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:07.928987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbaa39-4eee-4efa-88de-8267abdbd781 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.929271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbaa39-4eee-4efa-88de-8267abdbd781 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.929548 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbaa39-4eee-4efa-88de-8267abdbd781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.949499 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-87dbaa39-4eee-4efa-88de-8267abdbd781 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:07.950750 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-87dbaa39-4eee-4efa-88de-8267abdbd781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:07.969891 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1023/2018] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:07 2022] POST /identity/v3/users => generated 283 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:07.974412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73e3c1-b534-492c-bc8f-beabf307abc3 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:07.976274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1a73e3c1-b534-492c-bc8f-beabf307abc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:07.989416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a73e3c1-b534-492c-bc8f-beabf307abc3 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:08.019714 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 996/2019] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:08.027514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbaa39-4eee-4efa-88de-8267abdbd781 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:08.028532 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-87dbaa39-4eee-4efa-88de-8267abdbd781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:08.049723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6a499-c667-474d-af0b-7a6050b7fbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:08.050348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6a499-c667-474d-af0b-7a6050b7fbc2 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:08.050632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6a499-c667-474d-af0b-7a6050b7fbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:08.050914 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6a499-c667-474d-af0b-7a6050b7fbc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:08.051574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6a499-c667-474d-af0b-7a6050b7fbc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:08.053178 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6a499-c667-474d-af0b-7a6050b7fbc2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:08.067187 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1024/2020] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:08.068746 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6a499-c667-474d-af0b-7a6050b7fbc2 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:08.083085 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6a499-c667-474d-af0b-7a6050b7fbc2 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:08.087798 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741f6a0-83cd-4a95-865a-1f211a9faba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:08.088441 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741f6a0-83cd-4a95-865a-1f211a9faba5 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:08.089454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741f6a0-83cd-4a95-865a-1f211a9faba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:08.089568 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741f6a0-83cd-4a95-865a-1f211a9faba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:08.090213 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741f6a0-83cd-4a95-865a-1f211a9faba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:08.091844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741f6a0-83cd-4a95-865a-1f211a9faba5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:08.094240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbaa39-4eee-4efa-88de-8267abdbd781 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:08.098081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d793c27-eb34-4989-b74e-40e3c4eab50f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:08.098699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d793c27-eb34-4989-b74e-40e3c4eab50f service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:08.099024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d793c27-eb34-4989-b74e-40e3c4eab50f service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:08.099281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d793c27-eb34-4989-b74e-40e3c4eab50f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:08.103130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1025/2021] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:08.107507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741f6a0-83cd-4a95-865a-1f211a9faba5 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:08.108870 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d793c27-eb34-4989-b74e-40e3c4eab50f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:08.110993 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d793c27-eb34-4989-b74e-40e3c4eab50f service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:08.117239 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741f6a0-83cd-4a95-865a-1f211a9faba5 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:08.120130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 997/2022] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:01:08.122137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64632222-681c-4a74-a9fb-c80bca73780a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:08.122837 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64632222-681c-4a74-a9fb-c80bca73780a admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:08.123141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64632222-681c-4a74-a9fb-c80bca73780a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:08.123412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64632222-681c-4a74-a9fb-c80bca73780a admin admin] PATH_INFO: `/v3/projects/eaef5f0790a141b38b93737486329b4b/users/789bf8c6941543f6ab3e9d0fd5d13f8b/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:08.140588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64632222-681c-4a74-a9fb-c80bca73780a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaef5f0790a141b38b93737486329b4b, user_id=789bf8c6941543f6ab3e9d0fd5d13f8b, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:08.143411 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64632222-681c-4a74-a9fb-c80bca73780a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:08.163127 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:08.216422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1026/2023] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:08 2022] PUT /identity/v3/projects/eaef5f0790a141b38b93737486329b4b/users/789bf8c6941543f6ab3e9d0fd5d13f8b/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:08.228064 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4781719-f58f-4323-ac4d-efc3c1b90355 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:08.228373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4781719-f58f-4323-ac4d-efc3c1b90355 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:08.228647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4781719-f58f-4323-ac4d-efc3c1b90355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:08.258943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b4781719-f58f-4323-ac4d-efc3c1b90355 None None] MFA Rules not processed for user `789bf8c6941543f6ab3e9d0fd5d13f8b`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:08.260167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b4781719-f58f-4323-ac4d-efc3c1b90355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:08.326729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 998/2024] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:08.331392 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64632222-681c-4a74-a9fb-c80bca73780a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:08.405555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64632222-681c-4a74-a9fb-c80bca73780a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:08.410418 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0a93a-b158-4c82-b248-91d5d7435017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:08.411053 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0a93a-b158-4c82-b248-91d5d7435017 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:08.411332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0a93a-b158-4c82-b248-91d5d7435017 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:08.411598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e0a93a-b158-4c82-b248-91d5d7435017 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:08.412247 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0a93a-b158-4c82-b248-91d5d7435017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:08.414199 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e0a93a-b158-4c82-b248-91d5d7435017 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:08.519452 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1027/2025] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:08 2022] POST /identity/v3/projects => generated 365 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:08.523604 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4781719-f58f-4323-ac4d-efc3c1b90355 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:08.535135 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4781719-f58f-4323-ac4d-efc3c1b90355 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:08.542092 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0a93a-b158-4c82-b248-91d5d7435017 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:08.544674 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-55e0a93a-b158-4c82-b248-91d5d7435017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:08.605760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f6d49-0c56-433a-9cad-b87040fc0fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:08.606575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f6d49-0c56-433a-9cad-b87040fc0fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:08.606882 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f6d49-0c56-433a-9cad-b87040fc0fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:08.607150 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f6d49-0c56-433a-9cad-b87040fc0fa7 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:08.607874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f6d49-0c56-433a-9cad-b87040fc0fa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:08.609904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f6d49-0c56-433a-9cad-b87040fc0fa7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:08.617320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e0a93a-b158-4c82-b248-91d5d7435017 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:08.623245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb19c03c-13ee-407d-b3b6-be10c6aafeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:08.623838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19c03c-13ee-407d-b3b6-be10c6aafeb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:08.624133 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19c03c-13ee-407d-b3b6-be10c6aafeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:08.624400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb19c03c-13ee-407d-b3b6-be10c6aafeb9 admin admin] PATH_INFO: `/v3/projects/b29ed660360d4164a58e4d2a1663c51b` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:08.625675 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb19c03c-13ee-407d-b3b6-be10c6aafeb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b29ed660360d4164a58e4d2a1663c51b)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:08.627579 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb19c03c-13ee-407d-b3b6-be10c6aafeb9 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:08.718269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 999/2026] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:08 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:08.722328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f6d49-0c56-433a-9cad-b87040fc0fa7 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:08.738453 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f6d49-0c56-433a-9cad-b87040fc0fa7 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:08.741764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59fb18-dbbc-4f63-9e46-a7510291c419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:08.742490 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59fb18-dbbc-4f63-9e46-a7510291c419 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:08.742828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59fb18-dbbc-4f63-9e46-a7510291c419 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:08.743193 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59fb18-dbbc-4f63-9e46-a7510291c419 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:08.743849 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59fb18-dbbc-4f63-9e46-a7510291c419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:08.745525 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59fb18-dbbc-4f63-9e46-a7510291c419 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:08.759508 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cb19c03c-13ee-407d-b3b6-be10c6aafeb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b29ed660360d4164a58e4d2a1663c51b', 'request_id': 'req-cb19c03c-13ee-407d-b3b6-be10c6aafeb9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:08.765231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1000/2027] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:08.769667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59fb18-dbbc-4f63-9e46-a7510291c419 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:08.782952 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59fb18-dbbc-4f63-9e46-a7510291c419 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:08.814644 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:08.849281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cb19c03c-13ee-407d-b3b6-be10c6aafeb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b29ed660360d4164a58e4d2a1663c51b', 'request_id': 'req-cb19c03c-13ee-407d-b3b6-be10c6aafeb9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:08.857471 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cb19c03c-13ee-407d-b3b6-be10c6aafeb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b29ed660360d4164a58e4d2a1663c51b', 'request_id': 'req-cb19c03c-13ee-407d-b3b6-be10c6aafeb9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:08.868886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98c253-9d7d-4b3d-9c62-4cb89e8676e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:08.869697 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98c253-9d7d-4b3d-9c62-4cb89e8676e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:08.871020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98c253-9d7d-4b3d-9c62-4cb89e8676e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:08.872801 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98c253-9d7d-4b3d-9c62-4cb89e8676e9 admin admin] PATH_INFO: `/v3/projects/d78cfc7e74b54422965184c97687ae1f/users/5d626688937143e281f5a395c1e00e5b/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:08.881594 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cb19c03c-13ee-407d-b3b6-be10c6aafeb9 admin admin] The token cache is being invalidate because project b29ed660360d4164a58e4d2a1663c51b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:08.882067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cb19c03c-13ee-407d-b3b6-be10c6aafeb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:08.882812 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cb19c03c-13ee-407d-b3b6-be10c6aafeb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:08.884754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1028/2028] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:08 2022] DELETE /identity/v3/projects/b29ed660360d4164a58e4d2a1663c51b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:08.891241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd0210-736b-43c0-8925-a050765ca1e5 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:08.891612 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd0210-736b-43c0-8925-a050765ca1e5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:08.891891 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd0210-736b-43c0-8925-a050765ca1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:08.895561 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98c253-9d7d-4b3d-9c62-4cb89e8676e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d78cfc7e74b54422965184c97687ae1f, user_id=5d626688937143e281f5a395c1e00e5b, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:08.898690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98c253-9d7d-4b3d-9c62-4cb89e8676e9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:08.919471 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1fcd0210-736b-43c0-8925-a050765ca1e5 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:08.920690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1fcd0210-736b-43c0-8925-a050765ca1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:08.928672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1029/2029] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:08.937050 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d343d-859e-4312-8e1a-93a83b2a520d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:08.940805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d343d-859e-4312-8e1a-93a83b2a520d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:08.940805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d343d-859e-4312-8e1a-93a83b2a520d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:08.957306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-517d343d-859e-4312-8e1a-93a83b2a520d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:08.958525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-517d343d-859e-4312-8e1a-93a83b2a520d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:08.963354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1001/2030] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:08 2022] PUT /identity/v3/projects/d78cfc7e74b54422965184c97687ae1f/users/5d626688937143e281f5a395c1e00e5b/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:08.967251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98c253-9d7d-4b3d-9c62-4cb89e8676e9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:08.968904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-cd98c253-9d7d-4b3d-9c62-4cb89e8676e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:09.021187 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1030/2031] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:09.025881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d343d-859e-4312-8e1a-93a83b2a520d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:09.032779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98c253-9d7d-4b3d-9c62-4cb89e8676e9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:09.034652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d343d-859e-4312-8e1a-93a83b2a520d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:09.037689 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a4f57-4696-4bbc-b5d4-6b9cfc4334f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:09.038299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a4f57-4696-4bbc-b5d4-6b9cfc4334f2 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:09.038580 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a4f57-4696-4bbc-b5d4-6b9cfc4334f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:09.038858 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a4f57-4696-4bbc-b5d4-6b9cfc4334f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:09.039389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc02b2-8492-4f62-9427-afb7bdb350a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:09.043991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a4f57-4696-4bbc-b5d4-6b9cfc4334f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:09.044133 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc02b2-8492-4f62-9427-afb7bdb350a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:09.044133 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc02b2-8492-4f62-9427-afb7bdb350a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:09.044133 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc02b2-8492-4f62-9427-afb7bdb350a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:09.044133 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc02b2-8492-4f62-9427-afb7bdb350a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:09.044413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a4f57-4696-4bbc-b5d4-6b9cfc4334f2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:09.044484 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc02b2-8492-4f62-9427-afb7bdb350a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:09.044484 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc02b2-8492-4f62-9427-afb7bdb350a3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:09.052019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1002/2032] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:09.055829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1031/2033] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:01:09.056465 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a4f57-4696-4bbc-b5d4-6b9cfc4334f2 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:09.060325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc02b2-8492-4f62-9427-afb7bdb350a3 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:09.067802 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a4f57-4696-4bbc-b5d4-6b9cfc4334f2 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:09.070469 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc02b2-8492-4f62-9427-afb7bdb350a3 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:09.072751 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150c265-08fd-4750-aa44-7087ffd75dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:09.073445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150c265-08fd-4750-aa44-7087ffd75dba admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:09.073721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150c265-08fd-4750-aa44-7087ffd75dba admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:09.073994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150c265-08fd-4750-aa44-7087ffd75dba admin admin] PATH_INFO: `/v3/projects/d78cfc7e74b54422965184c97687ae1f/users/5d626688937143e281f5a395c1e00e5b/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:09.076071 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150c265-08fd-4750-aa44-7087ffd75dba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d78cfc7e74b54422965184c97687ae1f, user_id=5d626688937143e281f5a395c1e00e5b, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:09.076203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63e656-cbda-4bc4-af5c-1573333f0182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:09.076687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63e656-cbda-4bc4-af5c-1573333f0182 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:09.076964 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63e656-cbda-4bc4-af5c-1573333f0182 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:09.077235 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63e656-cbda-4bc4-af5c-1573333f0182 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:09.077883 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c63e656-cbda-4bc4-af5c-1573333f0182 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:09.078808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150c265-08fd-4750-aa44-7087ffd75dba admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:09.079890 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c63e656-cbda-4bc4-af5c-1573333f0182 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:09.089583 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1003/2034] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:09 2022] PUT /identity/v3/projects/d78cfc7e74b54422965184c97687ae1f/users/5d626688937143e281f5a395c1e00e5b/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:09.099915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a46db6-e21c-4fa9-9da8-e6f4f781e6d5 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:09.100195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a46db6-e21c-4fa9-9da8-e6f4f781e6d5 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:09.107246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a46db6-e21c-4fa9-9da8-e6f4f781e6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:09.130248 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-69a46db6-e21c-4fa9-9da8-e6f4f781e6d5 None None] MFA Rules not processed for user `5d626688937143e281f5a395c1e00e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:09.131501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-69a46db6-e21c-4fa9-9da8-e6f4f781e6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:09.193068 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1032/2035] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:09 2022] POST /identity/v3/projects => generated 357 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:09.197935 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63e656-cbda-4bc4-af5c-1573333f0182 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:09.202169 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1004/2036] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:09.219145 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a46db6-e21c-4fa9-9da8-e6f4f781e6d5 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:09.221147 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-69a46db6-e21c-4fa9-9da8-e6f4f781e6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:09.271558 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63e656-cbda-4bc4-af5c-1573333f0182 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:09.277140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3d5ec-9824-44ac-af86-b6118b63cfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:09.277744 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3d5ec-9824-44ac-af86-b6118b63cfc7 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:09.278020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3d5ec-9824-44ac-af86-b6118b63cfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:09.278289 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3d5ec-9824-44ac-af86-b6118b63cfc7 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:09.278933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3d5ec-9824-44ac-af86-b6118b63cfc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:09.280863 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3d5ec-9824-44ac-af86-b6118b63cfc7 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:09.281615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a46db6-e21c-4fa9-9da8-e6f4f781e6d5 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:09.332377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01c503-bcac-4795-91ce-6d1bc79629a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:09.332979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01c503-bcac-4795-91ce-6d1bc79629a5 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:09.333258 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01c503-bcac-4795-91ce-6d1bc79629a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:09.333525 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01c503-bcac-4795-91ce-6d1bc79629a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:09.364194 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1033/2037] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:09 2022] POST /identity/v3/users => generated 279 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:09.368339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3d5ec-9824-44ac-af86-b6118b63cfc7 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:09.379274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3d5ec-9824-44ac-af86-b6118b63cfc7 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:09.383904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa484e1-3b52-4294-a962-ea657b6a5ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:09.384509 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa484e1-3b52-4294-a962-ea657b6a5ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:09.384784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa484e1-3b52-4294-a962-ea657b6a5ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:09.385049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa484e1-3b52-4294-a962-ea657b6a5ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:09.385729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa484e1-3b52-4294-a962-ea657b6a5ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:09.387345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa484e1-3b52-4294-a962-ea657b6a5ad1 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:09.396238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01c503-bcac-4795-91ce-6d1bc79629a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:09.397709 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01c503-bcac-4795-91ce-6d1bc79629a5 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:09.398061 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1034/2038] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:09.402909 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa484e1-3b52-4294-a962-ea657b6a5ad1 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:09.410939 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1005/2039] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:01:09.413270 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa484e1-3b52-4294-a962-ea657b6a5ad1 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:09.418642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192eb934-eb70-4968-864c-73423dcb9492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:09.419371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192eb934-eb70-4968-864c-73423dcb9492 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:09.419662 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192eb934-eb70-4968-864c-73423dcb9492 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:09.419919 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192eb934-eb70-4968-864c-73423dcb9492 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:09.420563 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192eb934-eb70-4968-864c-73423dcb9492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:09.422185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192eb934-eb70-4968-864c-73423dcb9492 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:09.433143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1035/2040] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:09.438577 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01c503-bcac-4795-91ce-6d1bc79629a5 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:09.449367 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01c503-bcac-4795-91ce-6d1bc79629a5 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:09.454144 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d05367-9724-4363-b2c2-aa05611e4665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:09.454814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d05367-9724-4363-b2c2-aa05611e4665 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:09.455119 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d05367-9724-4363-b2c2-aa05611e4665 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:09.455411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d05367-9724-4363-b2c2-aa05611e4665 admin admin] PATH_INFO: `/v3/projects/b785c03c637e4faeb1495e11fb9c8332/users/d5b8a65a0b364c24a53a96a268bd5dea/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:09.470991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d05367-9724-4363-b2c2-aa05611e4665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b785c03c637e4faeb1495e11fb9c8332, user_id=d5b8a65a0b364c24a53a96a268bd5dea, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:09.473656 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d05367-9724-4363-b2c2-aa05611e4665 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:09.575493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1006/2041] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:09 2022] PUT /identity/v3/projects/b785c03c637e4faeb1495e11fb9c8332/users/d5b8a65a0b364c24a53a96a268bd5dea/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:09.589778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1f308-9b0c-46da-a277-69c2c1703326 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:09.590253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1f308-9b0c-46da-a277-69c2c1703326 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:09.590583 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1f308-9b0c-46da-a277-69c2c1703326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:09.619833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-98c1f308-9b0c-46da-a277-69c2c1703326 None None] MFA Rules not processed for user `d5b8a65a0b364c24a53a96a268bd5dea`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:09.621185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-98c1f308-9b0c-46da-a277-69c2c1703326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:09.649911 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:09.690939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1036/2042] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:09.700404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d05367-9724-4363-b2c2-aa05611e4665 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:09.725885 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1f308-9b0c-46da-a277-69c2c1703326 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:09.727646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-98c1f308-9b0c-46da-a277-69c2c1703326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:09.759635 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d05367-9724-4363-b2c2-aa05611e4665 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:09.764263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e887c2b-b7ea-488f-bf6d-a905cc98cab0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:09.764975 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e887c2b-b7ea-488f-bf6d-a905cc98cab0 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:09.765255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e887c2b-b7ea-488f-bf6d-a905cc98cab0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:09.765528 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e887c2b-b7ea-488f-bf6d-a905cc98cab0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:09.767613 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1e887c2b-b7ea-488f-bf6d-a905cc98cab0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:09.797397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1f308-9b0c-46da-a277-69c2c1703326 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:09.801308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485b1b2-5859-483d-a729-e532235f35e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:09.801989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485b1b2-5859-483d-a729-e532235f35e2 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:09.802343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485b1b2-5859-483d-a729-e532235f35e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:09.802651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485b1b2-5859-483d-a729-e532235f35e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:09.812287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485b1b2-5859-483d-a729-e532235f35e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:09.813836 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485b1b2-5859-483d-a729-e532235f35e2 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:09.823608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1037/2043] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:09 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:09.840368 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485b1b2-5859-483d-a729-e532235f35e2 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:09.845013 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e887c2b-b7ea-488f-bf6d-a905cc98cab0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:09.846463 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e887c2b-b7ea-488f-bf6d-a905cc98cab0 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:09.857065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1007/2044] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:09.932565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485b1b2-5859-483d-a729-e532235f35e2 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:09.938479 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7c931-bf86-42d0-887f-2841eabfa31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:09.939226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7c931-bf86-42d0-887f-2841eabfa31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:09.939522 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7c931-bf86-42d0-887f-2841eabfa31f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:09.939777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7c931-bf86-42d0-887f-2841eabfa31f admin admin] PATH_INFO: `/v3/users/d5b8a65a0b364c24a53a96a268bd5dea` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:09.941944 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a7c931-bf86-42d0-887f-2841eabfa31f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5b8a65a0b364c24a53a96a268bd5dea)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:09.943705 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a7c931-bf86-42d0-887f-2841eabfa31f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:10.020731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:10.217445 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d8a7c931-bf86-42d0-887f-2841eabfa31f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5b8a65a0b364c24a53a96a268bd5dea'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:10.311868 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d8a7c931-bf86-42d0-887f-2841eabfa31f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5b8a65a0b364c24a53a96a268bd5dea'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:10.312504 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d8a7c931-bf86-42d0-887f-2841eabfa31f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5b8a65a0b364c24a53a96a268bd5dea'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:10.321252 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d8a7c931-bf86-42d0-887f-2841eabfa31f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5b8a65a0b364c24a53a96a268bd5dea'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:10.333725 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d8a7c931-bf86-42d0-887f-2841eabfa31f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5b8a65a0b364c24a53a96a268bd5dea'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:10.341018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1038/2045] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:09 2022] DELETE /identity/v3/users/d5b8a65a0b364c24a53a96a268bd5dea => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:10.346468 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca89256-3ef1-4852-b329-30fc266959b3 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:10.346822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca89256-3ef1-4852-b329-30fc266959b3 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:10.347194 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca89256-3ef1-4852-b329-30fc266959b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:10.369490 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-dca89256-3ef1-4852-b329-30fc266959b3 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:10.370874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-dca89256-3ef1-4852-b329-30fc266959b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:10.435612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1008/2046] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:10.445873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7c931-bf86-42d0-887f-2841eabfa31f admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:10.447415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d8a7c931-bf86-42d0-887f-2841eabfa31f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:10.507145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7c931-bf86-42d0-887f-2841eabfa31f admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:10.511178 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7b3b2-90d8-4081-bb97-f9a11ef5375d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:10.511784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7b3b2-90d8-4081-bb97-f9a11ef5375d service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:10.512062 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7b3b2-90d8-4081-bb97-f9a11ef5375d service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:10.512335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7b3b2-90d8-4081-bb97-f9a11ef5375d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:10.521308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7b3b2-90d8-4081-bb97-f9a11ef5375d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:10.522743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7b3b2-90d8-4081-bb97-f9a11ef5375d service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:10.533409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1039/2047] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:10.586702 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:11.044904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca89256-3ef1-4852-b329-30fc266959b3 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:11.047022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-dca89256-3ef1-4852-b329-30fc266959b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:11.061843 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca89256-3ef1-4852-b329-30fc266959b3 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:11.068080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:11.068595 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:11.068937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:11.069294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14 admin admin] PATH_INFO: `/v3/projects/b785c03c637e4faeb1495e11fb9c8332` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:11.070857 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b785c03c637e4faeb1495e11fb9c8332)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:11.075556 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:11.240395 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b785c03c637e4faeb1495e11fb9c8332', 'request_id': 'req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:11.247652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b785c03c637e4faeb1495e11fb9c8332', 'request_id': 'req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:11.263729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b785c03c637e4faeb1495e11fb9c8332', 'request_id': 'req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:11.351271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14 admin admin] The token cache is being invalidate because project b785c03c637e4faeb1495e11fb9c8332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:11.351562 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:11.352224 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:11.353738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1009/2048] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:11 2022] DELETE /identity/v3/projects/b785c03c637e4faeb1495e11fb9c8332 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:11.947578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:12.203538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:12.837442 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:12.952002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:13.312005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:13.403210 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:13.484996 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:13.602938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:13.814214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:13.905903 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:13.935271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:14.106798 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:14.157574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:14.329252 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:14.357804 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:14.494487 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:14.523251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:14.609375 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:14.694858 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:14.723368 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:14.917658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:14.952112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:15.106928 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:15.187114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:15.425989 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:15.660845 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:15.838280 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:16.009415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:16.271930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:16.484753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:16.547106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:16.746973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:18.092878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7b3b2-90d8-4081-bb97-f9a11ef5375d service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:18.096035 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-cce7b3b2-90d8-4081-bb97-f9a11ef5375d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:18.120967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7b3b2-90d8-4081-bb97-f9a11ef5375d service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:18.196527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ee225-5611-4793-ae18-91d805a820ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:18.196527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ee225-5611-4793-ae18-91d805a820ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:18.196527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ee225-5611-4793-ae18-91d805a820ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:18.196527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ee225-5611-4793-ae18-91d805a820ee admin admin] PATH_INFO: `/v3/users/812de7a33ac94119a836b7dd39ea1813` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:18.197239 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ee225-5611-4793-ae18-91d805a820ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=812de7a33ac94119a836b7dd39ea1813)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:18.197239 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ee225-5611-4793-ae18-91d805a820ee admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:18.483748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c16ee225-5611-4793-ae18-91d805a820ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '812de7a33ac94119a836b7dd39ea1813'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:18.568876 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c16ee225-5611-4793-ae18-91d805a820ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '812de7a33ac94119a836b7dd39ea1813'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:18.569598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c16ee225-5611-4793-ae18-91d805a820ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '812de7a33ac94119a836b7dd39ea1813'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:18.577574 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c16ee225-5611-4793-ae18-91d805a820ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '812de7a33ac94119a836b7dd39ea1813'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:18.590619 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c16ee225-5611-4793-ae18-91d805a820ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '812de7a33ac94119a836b7dd39ea1813'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:18.597587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1040/2049] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:18 2022] DELETE /identity/v3/users/812de7a33ac94119a836b7dd39ea1813 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:18.848588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:18.850115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:18.864602 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f44e6b-cc7a-4007-aef7-dd6f77b8dd14 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:18.890715 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ee225-5611-4793-ae18-91d805a820ee admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:18.892369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c16ee225-5611-4793-ae18-91d805a820ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:18.953260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec925ec0-3e25-4af0-ae13-c0c2dd407d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:18.953965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec925ec0-3e25-4af0-ae13-c0c2dd407d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:18.954352 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec925ec0-3e25-4af0-ae13-c0c2dd407d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:18.954721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec925ec0-3e25-4af0-ae13-c0c2dd407d6b admin admin] PATH_INFO: `/v3/users/789bf8c6941543f6ab3e9d0fd5d13f8b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:18.956256 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec925ec0-3e25-4af0-ae13-c0c2dd407d6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=789bf8c6941543f6ab3e9d0fd5d13f8b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:18.958361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec925ec0-3e25-4af0-ae13-c0c2dd407d6b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:18.965656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:19.003382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ee225-5611-4793-ae18-91d805a820ee admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:19.008146 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4d11d-1e01-4800-9d29-65bce49a7445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:19.008791 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4d11d-1e01-4800-9d29-65bce49a7445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:19.009180 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4d11d-1e01-4800-9d29-65bce49a7445 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:19.009356 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4d11d-1e01-4800-9d29-65bce49a7445 admin admin] PATH_INFO: `/v3/projects/f36b9db4699345c9a54c39aa803e0984` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:19.010733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4d11d-1e01-4800-9d29-65bce49a7445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f36b9db4699345c9a54c39aa803e0984)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:19.012740 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4d11d-1e01-4800-9d29-65bce49a7445 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:19.250096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ec925ec0-3e25-4af0-ae13-c0c2dd407d6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '789bf8c6941543f6ab3e9d0fd5d13f8b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:19.251475 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-94b4d11d-1e01-4800-9d29-65bce49a7445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f36b9db4699345c9a54c39aa803e0984', 'request_id': 'req-94b4d11d-1e01-4800-9d29-65bce49a7445'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:19.322818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ec925ec0-3e25-4af0-ae13-c0c2dd407d6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '789bf8c6941543f6ab3e9d0fd5d13f8b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:19.332646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:19.338530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ec925ec0-3e25-4af0-ae13-c0c2dd407d6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '789bf8c6941543f6ab3e9d0fd5d13f8b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:19.339290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ec925ec0-3e25-4af0-ae13-c0c2dd407d6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '789bf8c6941543f6ab3e9d0fd5d13f8b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:19.347746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ec925ec0-3e25-4af0-ae13-c0c2dd407d6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '789bf8c6941543f6ab3e9d0fd5d13f8b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:19.354452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1010/2050] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:18 2022] DELETE /identity/v3/users/789bf8c6941543f6ab3e9d0fd5d13f8b => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:19.358771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711ba19-c1f0-4a30-8350-40888cbec2c8 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:19.359414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711ba19-c1f0-4a30-8350-40888cbec2c8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:19.359414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711ba19-c1f0-4a30-8350-40888cbec2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:19.379328 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-94b4d11d-1e01-4800-9d29-65bce49a7445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f36b9db4699345c9a54c39aa803e0984', 'request_id': 'req-94b4d11d-1e01-4800-9d29-65bce49a7445'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:19.380425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-6711ba19-c1f0-4a30-8350-40888cbec2c8 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:19.382179 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6711ba19-c1f0-4a30-8350-40888cbec2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:19.387764 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-94b4d11d-1e01-4800-9d29-65bce49a7445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f36b9db4699345c9a54c39aa803e0984', 'request_id': 'req-94b4d11d-1e01-4800-9d29-65bce49a7445'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:19.406528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-94b4d11d-1e01-4800-9d29-65bce49a7445 admin admin] The token cache is being invalidate because project f36b9db4699345c9a54c39aa803e0984 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:19.407093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-94b4d11d-1e01-4800-9d29-65bce49a7445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:19.407771 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-94b4d11d-1e01-4800-9d29-65bce49a7445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:19.409730 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1041/2051] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:18 2022] DELETE /identity/v3/projects/f36b9db4699345c9a54c39aa803e0984 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:19.413780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4d11d-1e01-4800-9d29-65bce49a7445 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:19.415196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-94b4d11d-1e01-4800-9d29-65bce49a7445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:19.459673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1011/2052] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:19.472122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711ba19-c1f0-4a30-8350-40888cbec2c8 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:19.474796 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6711ba19-c1f0-4a30-8350-40888cbec2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:19.526831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4d11d-1e01-4800-9d29-65bce49a7445 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:19.531978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954bac77-251c-417c-a2f4-3b9a2466c2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:19.532741 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bac77-251c-417c-a2f4-3b9a2466c2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:19.532898 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bac77-251c-417c-a2f4-3b9a2466c2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:19.533083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bac77-251c-417c-a2f4-3b9a2466c2cf admin admin] PATH_INFO: `/v3/users/d6be6f638e7a42628caba6e305d25014` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:19.534434 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954bac77-251c-417c-a2f4-3b9a2466c2cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6be6f638e7a42628caba6e305d25014)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:19.536430 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954bac77-251c-417c-a2f4-3b9a2466c2cf admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:19.583491 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711ba19-c1f0-4a30-8350-40888cbec2c8 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:19.591508 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4a633-c926-4768-8cd9-80e853d56cd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:19.592358 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4a633-c926-4768-8cd9-80e853d56cd2 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:19.592765 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4a633-c926-4768-8cd9-80e853d56cd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:19.593140 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4a633-c926-4768-8cd9-80e853d56cd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:19.607697 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4a633-c926-4768-8cd9-80e853d56cd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:19.610258 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4a633-c926-4768-8cd9-80e853d56cd2 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:19.624616 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1012/2053] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:01:19.786725 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-954bac77-251c-417c-a2f4-3b9a2466c2cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6be6f638e7a42628caba6e305d25014'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:19.866781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-954bac77-251c-417c-a2f4-3b9a2466c2cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6be6f638e7a42628caba6e305d25014'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:19.868827 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-954bac77-251c-417c-a2f4-3b9a2466c2cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6be6f638e7a42628caba6e305d25014'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:19.879295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-954bac77-251c-417c-a2f4-3b9a2466c2cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6be6f638e7a42628caba6e305d25014'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:19.887613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:19.895601 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-954bac77-251c-417c-a2f4-3b9a2466c2cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6be6f638e7a42628caba6e305d25014'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:19.903003 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1042/2054] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:19 2022] DELETE /identity/v3/users/d6be6f638e7a42628caba6e305d25014 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:20.313867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4a633-c926-4768-8cd9-80e853d56cd2 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:20.320766 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e4d4a633-c926-4768-8cd9-80e853d56cd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:20.321969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954bac77-251c-417c-a2f4-3b9a2466c2cf admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:20.323788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-954bac77-251c-417c-a2f4-3b9a2466c2cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:20.331727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4a633-c926-4768-8cd9-80e853d56cd2 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:20.401062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f6d2b-b959-44a2-b547-ecfe521c6e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:20.401710 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f6d2b-b959-44a2-b547-ecfe521c6e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:20.401999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f6d2b-b959-44a2-b547-ecfe521c6e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:20.402245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f6d2b-b959-44a2-b547-ecfe521c6e17 admin admin] PATH_INFO: `/v3/projects/eaef5f0790a141b38b93737486329b4b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:20.403706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f6d2b-b959-44a2-b547-ecfe521c6e17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaef5f0790a141b38b93737486329b4b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:20.407738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f6d2b-b959-44a2-b547-ecfe521c6e17 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:20.479551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954bac77-251c-417c-a2f4-3b9a2466c2cf admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:20.486215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422181b-377c-4c34-a635-8bd2068aca50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:20.486950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422181b-377c-4c34-a635-8bd2068aca50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:20.487240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422181b-377c-4c34-a635-8bd2068aca50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:20.487540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422181b-377c-4c34-a635-8bd2068aca50 admin admin] PATH_INFO: `/v3/projects/878db9335f1e48f99bb2d53513aedbf1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:20.489140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422181b-377c-4c34-a635-8bd2068aca50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=878db9335f1e48f99bb2d53513aedbf1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:20.490711 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422181b-377c-4c34-a635-8bd2068aca50 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:20.548809 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:20.579928 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d14f6d2b-b959-44a2-b547-ecfe521c6e17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaef5f0790a141b38b93737486329b4b', 'request_id': 'req-d14f6d2b-b959-44a2-b547-ecfe521c6e17'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:20.593988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d14f6d2b-b959-44a2-b547-ecfe521c6e17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaef5f0790a141b38b93737486329b4b', 'request_id': 'req-d14f6d2b-b959-44a2-b547-ecfe521c6e17'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:20.620938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d14f6d2b-b959-44a2-b547-ecfe521c6e17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaef5f0790a141b38b93737486329b4b', 'request_id': 'req-d14f6d2b-b959-44a2-b547-ecfe521c6e17'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:20.661297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d422181b-377c-4c34-a635-8bd2068aca50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '878db9335f1e48f99bb2d53513aedbf1', 'request_id': 'req-d422181b-377c-4c34-a635-8bd2068aca50'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:20.755761 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d14f6d2b-b959-44a2-b547-ecfe521c6e17 admin admin] The token cache is being invalidate because project eaef5f0790a141b38b93737486329b4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:20.756304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d14f6d2b-b959-44a2-b547-ecfe521c6e17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:20.757446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d14f6d2b-b959-44a2-b547-ecfe521c6e17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:20.759260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1013/2055] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:20 2022] DELETE /identity/v3/projects/eaef5f0790a141b38b93737486329b4b => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:20.763459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f6d2b-b959-44a2-b547-ecfe521c6e17 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:20.765250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d14f6d2b-b959-44a2-b547-ecfe521c6e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:20.835769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f6d2b-b959-44a2-b547-ecfe521c6e17 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:20.838739 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d422181b-377c-4c34-a635-8bd2068aca50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '878db9335f1e48f99bb2d53513aedbf1', 'request_id': 'req-d422181b-377c-4c34-a635-8bd2068aca50'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:20.856537 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d422181b-377c-4c34-a635-8bd2068aca50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '878db9335f1e48f99bb2d53513aedbf1', 'request_id': 'req-d422181b-377c-4c34-a635-8bd2068aca50'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:20.872347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d422181b-377c-4c34-a635-8bd2068aca50 admin admin] The token cache is being invalidate because project 878db9335f1e48f99bb2d53513aedbf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:20.872806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d422181b-377c-4c34-a635-8bd2068aca50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:20.873404 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d422181b-377c-4c34-a635-8bd2068aca50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:20.875240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1043/2056] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:20 2022] DELETE /identity/v3/projects/878db9335f1e48f99bb2d53513aedbf1 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:20.881303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5b658-df7f-41d9-9757-4e44205cd911 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:20.881683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5b658-df7f-41d9-9757-4e44205cd911 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:20.881868 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5b658-df7f-41d9-9757-4e44205cd911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:20.910060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-02d5b658-df7f-41d9-9757-4e44205cd911 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:20.911436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-02d5b658-df7f-41d9-9757-4e44205cd911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:20.919952 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465a155-66a1-4e24-8a14-cbfe8e9c9017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:20.921472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465a155-66a1-4e24-8a14-cbfe8e9c9017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:20.922306 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465a155-66a1-4e24-8a14-cbfe8e9c9017 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:20.923082 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465a155-66a1-4e24-8a14-cbfe8e9c9017 admin admin] PATH_INFO: `/v3/users/5d626688937143e281f5a395c1e00e5b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:20.924751 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465a155-66a1-4e24-8a14-cbfe8e9c9017 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d626688937143e281f5a395c1e00e5b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:20.928331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c465a155-66a1-4e24-8a14-cbfe8e9c9017 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:20.980617 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1044/2057] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:20.989401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bfb5b-fc39-4f48-a300-c2fe78c897c7 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:20.989688 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bfb5b-fc39-4f48-a300-c2fe78c897c7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:20.989964 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bfb5b-fc39-4f48-a300-c2fe78c897c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:21.010263 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-4a7bfb5b-fc39-4f48-a300-c2fe78c897c7 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:21.011519 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4a7bfb5b-fc39-4f48-a300-c2fe78c897c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:21.020016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1045/2058] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:21.026488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bfb5b-fc39-4f48-a300-c2fe78c897c7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:21.036355 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bfb5b-fc39-4f48-a300-c2fe78c897c7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:21.041765 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe4e15-18e2-4e4e-b4ed-c65513e99f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:21.042370 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe4e15-18e2-4e4e-b4ed-c65513e99f65 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:21.042645 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe4e15-18e2-4e4e-b4ed-c65513e99f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:21.042927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe4e15-18e2-4e4e-b4ed-c65513e99f65 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:21.043572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe4e15-18e2-4e4e-b4ed-c65513e99f65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:21.043868 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe4e15-18e2-4e4e-b4ed-c65513e99f65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:21.046037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe4e15-18e2-4e4e-b4ed-c65513e99f65 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:21.055348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:21.059142 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1046/2059] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:01:21.065049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe4e15-18e2-4e4e-b4ed-c65513e99f65 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:21.075550 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe4e15-18e2-4e4e-b4ed-c65513e99f65 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:21.080217 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab914cf3-b774-45bb-9cfd-9faf06a1315d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:21.080896 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab914cf3-b774-45bb-9cfd-9faf06a1315d admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:21.081162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab914cf3-b774-45bb-9cfd-9faf06a1315d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:21.081435 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab914cf3-b774-45bb-9cfd-9faf06a1315d admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:21.082072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab914cf3-b774-45bb-9cfd-9faf06a1315d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:21.084180 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab914cf3-b774-45bb-9cfd-9faf06a1315d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:21.213349 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c465a155-66a1-4e24-8a14-cbfe8e9c9017 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d626688937143e281f5a395c1e00e5b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:21.234178 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1047/2060] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:21 2022] POST /identity/v3/projects => generated 371 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:21.239672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab914cf3-b774-45bb-9cfd-9faf06a1315d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:21.282769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c465a155-66a1-4e24-8a14-cbfe8e9c9017 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d626688937143e281f5a395c1e00e5b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:21.292089 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:21.301128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c465a155-66a1-4e24-8a14-cbfe8e9c9017 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d626688937143e281f5a395c1e00e5b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:21.302083 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c465a155-66a1-4e24-8a14-cbfe8e9c9017 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d626688937143e281f5a395c1e00e5b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:21.311417 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c465a155-66a1-4e24-8a14-cbfe8e9c9017 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d626688937143e281f5a395c1e00e5b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:21.314714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab914cf3-b774-45bb-9cfd-9faf06a1315d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:21.318884 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1014/2061] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:20 2022] DELETE /identity/v3/users/5d626688937143e281f5a395c1e00e5b => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:21.320884 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486ad7a-7828-44c9-962e-a44ce0b3a4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:21.321861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486ad7a-7828-44c9-962e-a44ce0b3a4e2 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:21.322339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486ad7a-7828-44c9-962e-a44ce0b3a4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:21.322857 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486ad7a-7828-44c9-962e-a44ce0b3a4e2 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:21.323949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486ad7a-7828-44c9-962e-a44ce0b3a4e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:21.328163 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486ad7a-7828-44c9-962e-a44ce0b3a4e2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:21.615330 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1048/2062] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:21 2022] POST /identity/v3/users => generated 286 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:21.633273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465a155-66a1-4e24-8a14-cbfe8e9c9017 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:21.635031 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c465a155-66a1-4e24-8a14-cbfe8e9c9017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:21.707936 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c465a155-66a1-4e24-8a14-cbfe8e9c9017 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:21.713748 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5d131-f12a-4ece-90c8-1db23ab8a0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:21.714607 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5d131-f12a-4ece-90c8-1db23ab8a0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:21.714917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5d131-f12a-4ece-90c8-1db23ab8a0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:21.715198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5d131-f12a-4ece-90c8-1db23ab8a0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:21.715848 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5d131-f12a-4ece-90c8-1db23ab8a0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:21.717475 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5d131-f12a-4ece-90c8-1db23ab8a0a4 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:21.727816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1015/2063] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:21.732151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486ad7a-7828-44c9-962e-a44ce0b3a4e2 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:21.742636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486ad7a-7828-44c9-962e-a44ce0b3a4e2 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:21.748523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc92a259-d04e-4904-beed-3b20260eda26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:21.749132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92a259-d04e-4904-beed-3b20260eda26 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:21.749412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92a259-d04e-4904-beed-3b20260eda26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:21.749682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc92a259-d04e-4904-beed-3b20260eda26 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:21.750324 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92a259-d04e-4904-beed-3b20260eda26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:21.752011 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc92a259-d04e-4904-beed-3b20260eda26 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:21.762392 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1049/2064] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:21.767408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5d131-f12a-4ece-90c8-1db23ab8a0a4 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:21.778973 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5d131-f12a-4ece-90c8-1db23ab8a0a4 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:21.783760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe77e13-1047-4e43-b120-3b660fdb8f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:21.784531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe77e13-1047-4e43-b120-3b660fdb8f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:21.784730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe77e13-1047-4e43-b120-3b660fdb8f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:21.784998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe77e13-1047-4e43-b120-3b660fdb8f1a admin admin] PATH_INFO: `/v3/projects/4737f0a5e0164f309c591eba2a97ad0a/users/1187acb9ae484cbbb0451bdb6d8ff269/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:21.807731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe77e13-1047-4e43-b120-3b660fdb8f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4737f0a5e0164f309c591eba2a97ad0a, user_id=1187acb9ae484cbbb0451bdb6d8ff269, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:21.811774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe77e13-1047-4e43-b120-3b660fdb8f1a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:21.914331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1016/2065] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:21 2022] PUT /identity/v3/projects/4737f0a5e0164f309c591eba2a97ad0a/users/1187acb9ae484cbbb0451bdb6d8ff269/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:21.928480 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cddc7-63e2-4a51-9594-a496e97efd43 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:21.929912 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cddc7-63e2-4a51-9594-a496e97efd43 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:21.930381 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85cddc7-63e2-4a51-9594-a496e97efd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:21.967105 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d85cddc7-63e2-4a51-9594-a496e97efd43 None None] MFA Rules not processed for user `1187acb9ae484cbbb0451bdb6d8ff269`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:21.968348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d85cddc7-63e2-4a51-9594-a496e97efd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:22.047195 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1050/2066] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:22.063513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe77e13-1047-4e43-b120-3b660fdb8f1a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:22.065739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6fe77e13-1047-4e43-b120-3b660fdb8f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:22.069934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85cddc7-63e2-4a51-9594-a496e97efd43 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:22.073185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d85cddc7-63e2-4a51-9594-a496e97efd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:22.156482 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:22.176265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe77e13-1047-4e43-b120-3b660fdb8f1a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:22.179151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85cddc7-63e2-4a51-9594-a496e97efd43 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:22.180114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9217f8-bd0c-4799-9335-4459ecaebc0f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b060c70e1b440186eeac37d9c34567', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:22.180723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9217f8-bd0c-4799-9335-4459ecaebc0f service glance] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:22.180999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9217f8-bd0c-4799-9335-4459ecaebc0f service glance] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:22.181298 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9217f8-bd0c-4799-9335-4459ecaebc0f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:22.182833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466fbe4-1c53-40aa-856a-29d8915bf46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:22.186523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466fbe4-1c53-40aa-856a-29d8915bf46a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:22.186523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466fbe4-1c53-40aa-856a-29d8915bf46a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:22.186523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466fbe4-1c53-40aa-856a-29d8915bf46a admin admin] PATH_INFO: `/v3/projects/d78cfc7e74b54422965184c97687ae1f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:22.186523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466fbe4-1c53-40aa-856a-29d8915bf46a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d78cfc7e74b54422965184c97687ae1f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:22.187873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466fbe4-1c53-40aa-856a-29d8915bf46a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:22.192553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9217f8-bd0c-4799-9335-4459ecaebc0f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:22.194007 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9217f8-bd0c-4799-9335-4459ecaebc0f service glance] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:22.203298 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1017/2067] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:22.342296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9466fbe4-1c53-40aa-856a-29d8915bf46a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd78cfc7e74b54422965184c97687ae1f', 'request_id': 'req-9466fbe4-1c53-40aa-856a-29d8915bf46a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:22.431905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9466fbe4-1c53-40aa-856a-29d8915bf46a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd78cfc7e74b54422965184c97687ae1f', 'request_id': 'req-9466fbe4-1c53-40aa-856a-29d8915bf46a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:22.439779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9466fbe4-1c53-40aa-856a-29d8915bf46a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd78cfc7e74b54422965184c97687ae1f', 'request_id': 'req-9466fbe4-1c53-40aa-856a-29d8915bf46a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:22.451910 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9466fbe4-1c53-40aa-856a-29d8915bf46a admin admin] The token cache is being invalidate because project d78cfc7e74b54422965184c97687ae1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:22.458930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9466fbe4-1c53-40aa-856a-29d8915bf46a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:22.458930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9466fbe4-1c53-40aa-856a-29d8915bf46a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:22.458930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1051/2068] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:22 2022] DELETE /identity/v3/projects/d78cfc7e74b54422965184c97687ae1f => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:22.460463 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb42d9-15d4-4ac8-8bee-cfdadf7c2e8b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:22.460915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb42d9-15d4-4ac8-8bee-cfdadf7c2e8b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:22.461072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb42d9-15d4-4ac8-8bee-cfdadf7c2e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:22.492275 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-edeb42d9-15d4-4ac8-8bee-cfdadf7c2e8b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:22.493701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-edeb42d9-15d4-4ac8-8bee-cfdadf7c2e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:22.531019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:22.564506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1018/2069] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:22.572708 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3230747-099f-4742-90b1-68856fead317 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:22.573093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3230747-099f-4742-90b1-68856fead317 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:22.573392 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3230747-099f-4742-90b1-68856fead317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:22.595050 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d3230747-099f-4742-90b1-68856fead317 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:22.596370 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d3230747-099f-4742-90b1-68856fead317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:22.609172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1052/2070] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:22.613526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb42d9-15d4-4ac8-8bee-cfdadf7c2e8b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:22.625099 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb42d9-15d4-4ac8-8bee-cfdadf7c2e8b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:22.630065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9934e-fd71-4e86-b60c-3b908c819da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:22.630931 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9934e-fd71-4e86-b60c-3b908c819da2 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:22.631396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9934e-fd71-4e86-b60c-3b908c819da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:22.631679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9934e-fd71-4e86-b60c-3b908c819da2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:22.632814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9934e-fd71-4e86-b60c-3b908c819da2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:22.633378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9934e-fd71-4e86-b60c-3b908c819da2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:22.635496 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9934e-fd71-4e86-b60c-3b908c819da2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:22.648165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1019/2071] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:22.653089 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3230747-099f-4742-90b1-68856fead317 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:22.662489 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3230747-099f-4742-90b1-68856fead317 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:22.667248 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a34bae-2676-4df4-aa1f-dfa42d74ddf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:22.670021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a34bae-2676-4df4-aa1f-dfa42d74ddf1 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:22.670021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a34bae-2676-4df4-aa1f-dfa42d74ddf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:22.670021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a34bae-2676-4df4-aa1f-dfa42d74ddf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:22.670021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a34bae-2676-4df4-aa1f-dfa42d74ddf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:22.671425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a34bae-2676-4df4-aa1f-dfa42d74ddf1 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:22.800524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1053/2072] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:22 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:22.805292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9934e-fd71-4e86-b60c-3b908c819da2 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:22.816280 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9934e-fd71-4e86-b60c-3b908c819da2 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:22.882601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b57877-4cbc-4663-b3e3-809edfc3a16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:22.883124 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b57877-4cbc-4663-b3e3-809edfc3a16a admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:22.883396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b57877-4cbc-4663-b3e3-809edfc3a16a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:22.883664 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b57877-4cbc-4663-b3e3-809edfc3a16a admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:22.884437 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b57877-4cbc-4663-b3e3-809edfc3a16a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:22.886355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b57877-4cbc-4663-b3e3-809edfc3a16a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:23.023204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1020/2073] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:22 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:23.034791 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a34bae-2676-4df4-aa1f-dfa42d74ddf1 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:23.046608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a34bae-2676-4df4-aa1f-dfa42d74ddf1 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:23.051422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b115a-d7a4-4dd9-b255-d66e6b636829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:23.052018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b115a-d7a4-4dd9-b255-d66e6b636829 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:23.052297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b115a-d7a4-4dd9-b255-d66e6b636829 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:23.052592 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717b115a-d7a4-4dd9-b255-d66e6b636829 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:23.053223 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b115a-d7a4-4dd9-b255-d66e6b636829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:23.054845 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717b115a-d7a4-4dd9-b255-d66e6b636829 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:23.065894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1054/2074] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:23.070145 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b57877-4cbc-4663-b3e3-809edfc3a16a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:23.081394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b57877-4cbc-4663-b3e3-809edfc3a16a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:23.086457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac60256-b81b-4557-b179-120927b3f836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:23.087367 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac60256-b81b-4557-b179-120927b3f836 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:23.087695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac60256-b81b-4557-b179-120927b3f836 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:23.087992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac60256-b81b-4557-b179-120927b3f836 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:23.088727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac60256-b81b-4557-b179-120927b3f836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:23.090389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac60256-b81b-4557-b179-120927b3f836 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:23.103861 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1021/2075] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:23.112343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b115a-d7a4-4dd9-b255-d66e6b636829 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:23.126326 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717b115a-d7a4-4dd9-b255-d66e6b636829 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:23.131277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bba231-e5a8-4efe-9674-22b6742f2e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:23.132084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bba231-e5a8-4efe-9674-22b6742f2e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:23.132472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bba231-e5a8-4efe-9674-22b6742f2e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:23.133309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bba231-e5a8-4efe-9674-22b6742f2e7b admin admin] PATH_INFO: `/v3/projects/67391911bf8e48d8951141fde1346318/users/bf1461309da5404aa57278ce6f5867b1/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:23.160333 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bba231-e5a8-4efe-9674-22b6742f2e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=67391911bf8e48d8951141fde1346318, user_id=bf1461309da5404aa57278ce6f5867b1, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:23.168397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bba231-e5a8-4efe-9674-22b6742f2e7b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:23.264710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1055/2076] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:23 2022] PUT /identity/v3/projects/67391911bf8e48d8951141fde1346318/users/bf1461309da5404aa57278ce6f5867b1/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:23.282340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fd617-f954-44b8-9052-6448654a9410 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:23.282708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fd617-f954-44b8-9052-6448654a9410 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:23.283130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fd617-f954-44b8-9052-6448654a9410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:23.323420 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c63fd617-f954-44b8-9052-6448654a9410 None None] MFA Rules not processed for user `bf1461309da5404aa57278ce6f5867b1`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:23.324806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c63fd617-f954-44b8-9052-6448654a9410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:23.419523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1022/2077] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:23.430729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bba231-e5a8-4efe-9674-22b6742f2e7b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:23.432872 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-85bba231-e5a8-4efe-9674-22b6742f2e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:23.499852 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bba231-e5a8-4efe-9674-22b6742f2e7b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:23.503707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc662cb-45b9-410e-80cb-269fbb8bd1f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:23.505660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc662cb-45b9-410e-80cb-269fbb8bd1f5 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:23.505660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc662cb-45b9-410e-80cb-269fbb8bd1f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:23.505660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc662cb-45b9-410e-80cb-269fbb8bd1f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:23.515022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc662cb-45b9-410e-80cb-269fbb8bd1f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:23.516523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc662cb-45b9-410e-80cb-269fbb8bd1f5 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:23.525940 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1056/2078] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:33.318397 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fd617-f954-44b8-9052-6448654a9410 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:33.320605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c63fd617-f954-44b8-9052-6448654a9410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:33.374255 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:33.417646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fd617-f954-44b8-9052-6448654a9410 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:33.422588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cdae1-40ba-47d4-a5a7-25bfb4c135b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:33.423235 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cdae1-40ba-47d4-a5a7-25bfb4c135b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:33.423516 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cdae1-40ba-47d4-a5a7-25bfb4c135b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:33.423797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63cdae1-40ba-47d4-a5a7-25bfb4c135b6 admin admin] PATH_INFO: `/v3/users/5f91bc2624a248ec95aec5ca3233a8bc` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:33.424955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cdae1-40ba-47d4-a5a7-25bfb4c135b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f91bc2624a248ec95aec5ca3233a8bc)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:33.426883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63cdae1-40ba-47d4-a5a7-25bfb4c135b6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:33.521811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:33.692492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e63cdae1-40ba-47d4-a5a7-25bfb4c135b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f91bc2624a248ec95aec5ca3233a8bc'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:33.809030 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e63cdae1-40ba-47d4-a5a7-25bfb4c135b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f91bc2624a248ec95aec5ca3233a8bc'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:33.829579 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e63cdae1-40ba-47d4-a5a7-25bfb4c135b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f91bc2624a248ec95aec5ca3233a8bc'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:33.830842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e63cdae1-40ba-47d4-a5a7-25bfb4c135b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f91bc2624a248ec95aec5ca3233a8bc'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:33.847163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e63cdae1-40ba-47d4-a5a7-25bfb4c135b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f91bc2624a248ec95aec5ca3233a8bc'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:33.853689 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1023/2079] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:33 2022] DELETE /identity/v3/users/5f91bc2624a248ec95aec5ca3233a8bc => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:33.858402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c92de8-606d-4d50-8364-562c120792f7 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:33.858683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c92de8-606d-4d50-8364-562c120792f7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:33.859013 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c92de8-606d-4d50-8364-562c120792f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:33.878433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-00c92de8-606d-4d50-8364-562c120792f7 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:33.880200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-00c92de8-606d-4d50-8364-562c120792f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:33.933421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cdae1-40ba-47d4-a5a7-25bfb4c135b6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:33.934823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e63cdae1-40ba-47d4-a5a7-25bfb4c135b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:33.946386 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1057/2080] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:33.961689 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c92de8-606d-4d50-8364-562c120792f7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:33.963790 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-00c92de8-606d-4d50-8364-562c120792f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:34.057414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63cdae1-40ba-47d4-a5a7-25bfb4c135b6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:34.072109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c92de8-606d-4d50-8364-562c120792f7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:34.073280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a5df8-0676-45bd-845e-ec9c242c097f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:34.074249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a5df8-0676-45bd-845e-ec9c242c097f service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:34.074249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a5df8-0676-45bd-845e-ec9c242c097f service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:34.074600 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a5df8-0676-45bd-845e-ec9c242c097f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:34.079523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ef6cc-4aec-4128-9a56-03911ebb24c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:34.082294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ef6cc-4aec-4128-9a56-03911ebb24c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:34.083629 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ef6cc-4aec-4128-9a56-03911ebb24c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:34.085423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ef6cc-4aec-4128-9a56-03911ebb24c9 admin admin] PATH_INFO: `/v3/users/bf1461309da5404aa57278ce6f5867b1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:34.089783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ef6cc-4aec-4128-9a56-03911ebb24c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf1461309da5404aa57278ce6f5867b1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:34.100472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ef6cc-4aec-4128-9a56-03911ebb24c9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:34.111886 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a5df8-0676-45bd-845e-ec9c242c097f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:34.114578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a5df8-0676-45bd-845e-ec9c242c097f service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:34.130212 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1058/2081] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:01:34.326233 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cb0ef6cc-4aec-4128-9a56-03911ebb24c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf1461309da5404aa57278ce6f5867b1'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:34.335650 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:34.420677 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cb0ef6cc-4aec-4128-9a56-03911ebb24c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf1461309da5404aa57278ce6f5867b1'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:34.435745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cb0ef6cc-4aec-4128-9a56-03911ebb24c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf1461309da5404aa57278ce6f5867b1'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:34.436490 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cb0ef6cc-4aec-4128-9a56-03911ebb24c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf1461309da5404aa57278ce6f5867b1'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:34.444067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cb0ef6cc-4aec-4128-9a56-03911ebb24c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf1461309da5404aa57278ce6f5867b1'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:34.452097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1024/2082] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:33 2022] DELETE /identity/v3/users/bf1461309da5404aa57278ce6f5867b1 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:34.460564 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded2eae-0c72-4cd8-bb57-2e8887c66fe1 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:34.461710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded2eae-0c72-4cd8-bb57-2e8887c66fe1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:34.461987 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded2eae-0c72-4cd8-bb57-2e8887c66fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:34.493401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-8ded2eae-0c72-4cd8-bb57-2e8887c66fe1 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:34.494713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8ded2eae-0c72-4cd8-bb57-2e8887c66fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:34.605990 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1059/2083] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:34.618622 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ef6cc-4aec-4128-9a56-03911ebb24c9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:34.620788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-cb0ef6cc-4aec-4128-9a56-03911ebb24c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:34.731299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ef6cc-4aec-4128-9a56-03911ebb24c9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:34.735305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299ae68-ed78-4a6f-a6ff-5f81766d048e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:34.735914 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299ae68-ed78-4a6f-a6ff-5f81766d048e service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:34.736193 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299ae68-ed78-4a6f-a6ff-5f81766d048e service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:34.745384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299ae68-ed78-4a6f-a6ff-5f81766d048e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:34.748430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299ae68-ed78-4a6f-a6ff-5f81766d048e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:34.750351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299ae68-ed78-4a6f-a6ff-5f81766d048e service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:34.768466 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1025/2084] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:01:34.770644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded2eae-0c72-4cd8-bb57-2e8887c66fe1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:34.772568 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8ded2eae-0c72-4cd8-bb57-2e8887c66fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:34.791096 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded2eae-0c72-4cd8-bb57-2e8887c66fe1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:34.812380 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9171963-2143-45e6-8dd4-8e6557142c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:34.813388 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9171963-2143-45e6-8dd4-8e6557142c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:34.813664 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9171963-2143-45e6-8dd4-8e6557142c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:34.814057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9171963-2143-45e6-8dd4-8e6557142c3d admin admin] PATH_INFO: `/v3/projects/f62d7c95cfb144009114cf09d5d0319e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:34.817761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9171963-2143-45e6-8dd4-8e6557142c3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f62d7c95cfb144009114cf09d5d0319e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:34.820386 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9171963-2143-45e6-8dd4-8e6557142c3d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:34.991158 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:35.008017 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e9171963-2143-45e6-8dd4-8e6557142c3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f62d7c95cfb144009114cf09d5d0319e', 'request_id': 'req-e9171963-2143-45e6-8dd4-8e6557142c3d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:35.146226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e9171963-2143-45e6-8dd4-8e6557142c3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f62d7c95cfb144009114cf09d5d0319e', 'request_id': 'req-e9171963-2143-45e6-8dd4-8e6557142c3d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:35.160374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e9171963-2143-45e6-8dd4-8e6557142c3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f62d7c95cfb144009114cf09d5d0319e', 'request_id': 'req-e9171963-2143-45e6-8dd4-8e6557142c3d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:35.194016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e9171963-2143-45e6-8dd4-8e6557142c3d admin admin] The token cache is being invalidate because project f62d7c95cfb144009114cf09d5d0319e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:35.205050 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e9171963-2143-45e6-8dd4-8e6557142c3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:35.209683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e9171963-2143-45e6-8dd4-8e6557142c3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:35.212117 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1060/2085] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:34 2022] DELETE /identity/v3/projects/f62d7c95cfb144009114cf09d5d0319e => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:35.217741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6772089-d8dc-4981-a6c6-be295116a4f2 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.218051 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6772089-d8dc-4981-a6c6-be295116a4f2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:35.218342 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6772089-d8dc-4981-a6c6-be295116a4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:35.245441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f6772089-d8dc-4981-a6c6-be295116a4f2 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:35.247103 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f6772089-d8dc-4981-a6c6-be295116a4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:35.349724 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1026/2086] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:35.358637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ecf15d-f64d-42bf-99c2-91c9b634c028 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.359289 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ecf15d-f64d-42bf-99c2-91c9b634c028 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:35.360414 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ecf15d-f64d-42bf-99c2-91c9b634c028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:35.397748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-67ecf15d-f64d-42bf-99c2-91c9b634c028 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:35.400482 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-67ecf15d-f64d-42bf-99c2-91c9b634c028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:35.410961 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1061/2087] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:35.416311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6772089-d8dc-4981-a6c6-be295116a4f2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:35.428267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6772089-d8dc-4981-a6c6-be295116a4f2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:35.439524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee727024-c76e-401c-bba5-006d3398f96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:35.440370 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee727024-c76e-401c-bba5-006d3398f96e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.447226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ecf15d-f64d-42bf-99c2-91c9b634c028 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:35.447493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee727024-c76e-401c-bba5-006d3398f96e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:35.447493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee727024-c76e-401c-bba5-006d3398f96e admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:35.447706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-67ecf15d-f64d-42bf-99c2-91c9b634c028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:35.447791 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee727024-c76e-401c-bba5-006d3398f96e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:35.447791 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee727024-c76e-401c-bba5-006d3398f96e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:35.447791 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee727024-c76e-401c-bba5-006d3398f96e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:35.459761 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1027/2088] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:35.466813 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee727024-c76e-401c-bba5-006d3398f96e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:35.467029 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ecf15d-f64d-42bf-99c2-91c9b634c028 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:35.471858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958892fb-c863-42e7-8ed8-c400ee313e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:35.472497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958892fb-c863-42e7-8ed8-c400ee313e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.472774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958892fb-c863-42e7-8ed8-c400ee313e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:35.473044 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958892fb-c863-42e7-8ed8-c400ee313e0c admin admin] PATH_INFO: `/v3/projects/67391911bf8e48d8951141fde1346318` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:35.474170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958892fb-c863-42e7-8ed8-c400ee313e0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=67391911bf8e48d8951141fde1346318)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:35.489851 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958892fb-c863-42e7-8ed8-c400ee313e0c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:35.491407 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee727024-c76e-401c-bba5-006d3398f96e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:35.497295 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd542c-fd7e-47c7-bcfd-f2a29858ace8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:35.497924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd542c-fd7e-47c7-bcfd-f2a29858ace8 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.498240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd542c-fd7e-47c7-bcfd-f2a29858ace8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:35.498514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd542c-fd7e-47c7-bcfd-f2a29858ace8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:35.499728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd542c-fd7e-47c7-bcfd-f2a29858ace8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:35.503405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd542c-fd7e-47c7-bcfd-f2a29858ace8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:35.623239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1028/2089] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:35 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:35.628763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd542c-fd7e-47c7-bcfd-f2a29858ace8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:35.658169 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-958892fb-c863-42e7-8ed8-c400ee313e0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67391911bf8e48d8951141fde1346318', 'request_id': 'req-958892fb-c863-42e7-8ed8-c400ee313e0c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:35.699435 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:35.761662 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd542c-fd7e-47c7-bcfd-f2a29858ace8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:35.768733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-958892fb-c863-42e7-8ed8-c400ee313e0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67391911bf8e48d8951141fde1346318', 'request_id': 'req-958892fb-c863-42e7-8ed8-c400ee313e0c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:35.781473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-958892fb-c863-42e7-8ed8-c400ee313e0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67391911bf8e48d8951141fde1346318', 'request_id': 'req-958892fb-c863-42e7-8ed8-c400ee313e0c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:35.794217 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-958892fb-c863-42e7-8ed8-c400ee313e0c admin admin] The token cache is being invalidate because project 67391911bf8e48d8951141fde1346318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:35.794873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-958892fb-c863-42e7-8ed8-c400ee313e0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:35.795795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-958892fb-c863-42e7-8ed8-c400ee313e0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:35.797701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1062/2090] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:35 2022] DELETE /identity/v3/projects/67391911bf8e48d8951141fde1346318 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:35.803743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf5f74-328d-41c8-a54b-f2745719219d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.804035 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf5f74-328d-41c8-a54b-f2745719219d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:35.804601 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf5f74-328d-41c8-a54b-f2745719219d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:35.833149 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-fbdf5f74-328d-41c8-a54b-f2745719219d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:35.834797 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fbdf5f74-328d-41c8-a54b-f2745719219d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:35.842533 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffccbd-3c99-4bc9-b5c5-828c521005ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:35.843663 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffccbd-3c99-4bc9-b5c5-828c521005ba admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.852514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffccbd-3c99-4bc9-b5c5-828c521005ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:35.852514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ffccbd-3c99-4bc9-b5c5-828c521005ba admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:35.852514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffccbd-3c99-4bc9-b5c5-828c521005ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:35.852514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ffccbd-3c99-4bc9-b5c5-828c521005ba admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:35.905382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1063/2091] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:35.914101 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881680d9-b3f4-48fd-98d2-7a3938357e65 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.914381 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881680d9-b3f4-48fd-98d2-7a3938357e65 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:35.914649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881680d9-b3f4-48fd-98d2-7a3938357e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:35.935515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-881680d9-b3f4-48fd-98d2-7a3938357e65 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:35.937788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-881680d9-b3f4-48fd-98d2-7a3938357e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:35.945557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1064/2092] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:35.950511 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881680d9-b3f4-48fd-98d2-7a3938357e65 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:35.960778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881680d9-b3f4-48fd-98d2-7a3938357e65 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:35.964647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1029/2093] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:35 2022] POST /identity/v3/users => generated 286 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:35.971521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156126d3-8273-439e-9c51-552c9890df5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:35.972290 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156126d3-8273-439e-9c51-552c9890df5a admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.972429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffccbd-3c99-4bc9-b5c5-828c521005ba admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:35.972601 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156126d3-8273-439e-9c51-552c9890df5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:35.972967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156126d3-8273-439e-9c51-552c9890df5a admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:35.973537 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156126d3-8273-439e-9c51-552c9890df5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:35.974373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156126d3-8273-439e-9c51-552c9890df5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:35.974486 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c3ffccbd-3c99-4bc9-b5c5-828c521005ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:35.976124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156126d3-8273-439e-9c51-552c9890df5a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:35.990351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ffccbd-3c99-4bc9-b5c5-828c521005ba admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:35.991212 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1065/2094] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:01:35.992010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156126d3-8273-439e-9c51-552c9890df5a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:35.994174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-156126d3-8273-439e-9c51-552c9890df5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:35.999019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77784d23-28ee-4082-8cba-ba1c0095f7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:35.999642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77784d23-28ee-4082-8cba-ba1c0095f7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.999923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77784d23-28ee-4082-8cba-ba1c0095f7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.000191 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77784d23-28ee-4082-8cba-ba1c0095f7c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.000882 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77784d23-28ee-4082-8cba-ba1c0095f7c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:36.002511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77784d23-28ee-4082-8cba-ba1c0095f7c7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:36.008016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156126d3-8273-439e-9c51-552c9890df5a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:36.013709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec4b24-976c-47c4-9a82-c3fda83de3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:36.017908 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec4b24-976c-47c4-9a82-c3fda83de3a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.018075 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1030/2095] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:36.018183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec4b24-976c-47c4-9a82-c3fda83de3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.018183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec4b24-976c-47c4-9a82-c3fda83de3a7 admin admin] PATH_INFO: `/v3/users/5771e9d0c5c94306b3ec571916acdcaa` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.018478 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77784d23-28ee-4082-8cba-ba1c0095f7c7 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:36.019469 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec4b24-976c-47c4-9a82-c3fda83de3a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5771e9d0c5c94306b3ec571916acdcaa)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:36.021588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec4b24-976c-47c4-9a82-c3fda83de3a7 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:36.031377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77784d23-28ee-4082-8cba-ba1c0095f7c7 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:36.037312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4fd9b2-483c-4160-8807-b24a76b2770d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:36.038156 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fd9b2-483c-4160-8807-b24a76b2770d admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.038282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fd9b2-483c-4160-8807-b24a76b2770d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.038487 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fd9b2-483c-4160-8807-b24a76b2770d admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.039161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4fd9b2-483c-4160-8807-b24a76b2770d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:36.041069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4fd9b2-483c-4160-8807-b24a76b2770d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:36.051345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:36.164865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1031/2096] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:36 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:36.165874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4fd9b2-483c-4160-8807-b24a76b2770d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:36.231289 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4fd9b2-483c-4160-8807-b24a76b2770d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:36.236745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8790e1b-bea4-4bf9-b5bb-29d045c6c09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:36.237945 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8790e1b-bea4-4bf9-b5bb-29d045c6c09c admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.238580 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8790e1b-bea4-4bf9-b5bb-29d045c6c09c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.238869 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8790e1b-bea4-4bf9-b5bb-29d045c6c09c admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.239846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8790e1b-bea4-4bf9-b5bb-29d045c6c09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:36.241311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8790e1b-bea4-4bf9-b5bb-29d045c6c09c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:36.258784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1032/2097] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:36.258784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8790e1b-bea4-4bf9-b5bb-29d045c6c09c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:36.260562 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b9ec4b24-976c-47c4-9a82-c3fda83de3a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5771e9d0c5c94306b3ec571916acdcaa'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:36.268113 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8790e1b-bea4-4bf9-b5bb-29d045c6c09c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:36.273237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837cfd6-5431-4d35-ab87-6855c060c4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:36.275162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837cfd6-5431-4d35-ab87-6855c060c4b6 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.275162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837cfd6-5431-4d35-ab87-6855c060c4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.275162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837cfd6-5431-4d35-ab87-6855c060c4b6 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.275422 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837cfd6-5431-4d35-ab87-6855c060c4b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:36.277258 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6837cfd6-5431-4d35-ab87-6855c060c4b6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:36.414143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b9ec4b24-976c-47c4-9a82-c3fda83de3a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5771e9d0c5c94306b3ec571916acdcaa'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:36.414769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b9ec4b24-976c-47c4-9a82-c3fda83de3a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5771e9d0c5c94306b3ec571916acdcaa'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:36.421350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1033/2098] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:36 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:36.421853 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837cfd6-5431-4d35-ab87-6855c060c4b6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:36.423269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6837cfd6-5431-4d35-ab87-6855c060c4b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:36.425028 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b9ec4b24-976c-47c4-9a82-c3fda83de3a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5771e9d0c5c94306b3ec571916acdcaa'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:36.436441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837cfd6-5431-4d35-ab87-6855c060c4b6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:36.440465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b9ec4b24-976c-47c4-9a82-c3fda83de3a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5771e9d0c5c94306b3ec571916acdcaa'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:36.442318 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d08ec-e4a6-48ea-9f47-998ba03c3347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:36.443098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d08ec-e4a6-48ea-9f47-998ba03c3347 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.456642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d08ec-e4a6-48ea-9f47-998ba03c3347 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.456642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d08ec-e4a6-48ea-9f47-998ba03c3347 admin admin] PATH_INFO: `/v3/projects/c10d9bc9c0724621b7be95f79852bf0c/users/cf3eb99c308142388e527733ec12516a/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.456853 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1066/2099] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:35 2022] DELETE /identity/v3/users/5771e9d0c5c94306b3ec571916acdcaa => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:36.456853 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec4b24-976c-47c4-9a82-c3fda83de3a7 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:36.456853 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b9ec4b24-976c-47c4-9a82-c3fda83de3a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:36.462089 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d08ec-e4a6-48ea-9f47-998ba03c3347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c10d9bc9c0724621b7be95f79852bf0c, user_id=cf3eb99c308142388e527733ec12516a, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:36.465242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d08ec-e4a6-48ea-9f47-998ba03c3347 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:36.538173 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1034/2100] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:36 2022] PUT /identity/v3/projects/c10d9bc9c0724621b7be95f79852bf0c/users/cf3eb99c308142388e527733ec12516a/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:36.543052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec4b24-976c-47c4-9a82-c3fda83de3a7 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:36.555787 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8979aa0-6b32-42bf-b0ff-e1a83e064cda None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.556221 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8979aa0-6b32-42bf-b0ff-e1a83e064cda None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.556715 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8979aa0-6b32-42bf-b0ff-e1a83e064cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.591962 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f8979aa0-6b32-42bf-b0ff-e1a83e064cda None None] MFA Rules not processed for user `cf3eb99c308142388e527733ec12516a`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:36.593340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f8979aa0-6b32-42bf-b0ff-e1a83e064cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:36.646953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1440a-240b-4112-8501-96f40ecf9ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:36.646953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1440a-240b-4112-8501-96f40ecf9ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.646953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1440a-240b-4112-8501-96f40ecf9ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.646953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1440a-240b-4112-8501-96f40ecf9ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.648194 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1440a-240b-4112-8501-96f40ecf9ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:36.650306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1440a-240b-4112-8501-96f40ecf9ee0 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:36.682543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1067/2101] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:36.682543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1440a-240b-4112-8501-96f40ecf9ee0 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:36.693669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1440a-240b-4112-8501-96f40ecf9ee0 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:36.702352 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1035/2102] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:36.702549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dd0da-62e5-4ee8-9087-91d42bea7d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:36.703305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dd0da-62e5-4ee8-9087-91d42bea7d4d admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.704280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dd0da-62e5-4ee8-9087-91d42bea7d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.704687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dd0da-62e5-4ee8-9087-91d42bea7d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.705568 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629dd0da-62e5-4ee8-9087-91d42bea7d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:36.707591 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629dd0da-62e5-4ee8-9087-91d42bea7d4d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:36.718151 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8979aa0-6b32-42bf-b0ff-e1a83e064cda None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:36.718151 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:36.722647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f8979aa0-6b32-42bf-b0ff-e1a83e064cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:36.722959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1068/2103] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:36.727024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dd0da-62e5-4ee8-9087-91d42bea7d4d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:36.748958 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dd0da-62e5-4ee8-9087-91d42bea7d4d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:36.755851 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db4455-0991-440e-bb96-9758cdf4cd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:36.756957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db4455-0991-440e-bb96-9758cdf4cd44 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.756957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db4455-0991-440e-bb96-9758cdf4cd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.757192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db4455-0991-440e-bb96-9758cdf4cd44 admin admin] PATH_INFO: `/v3/projects/d572b7ff07cd47f18a2c2db1b9877804/users/c8b2be0d1e4b4456af201cf36eaf066b/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.778506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db4455-0991-440e-bb96-9758cdf4cd44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d572b7ff07cd47f18a2c2db1b9877804, user_id=c8b2be0d1e4b4456af201cf36eaf066b, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:36.782859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db4455-0991-440e-bb96-9758cdf4cd44 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:36.806775 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8979aa0-6b32-42bf-b0ff-e1a83e064cda None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:36.810923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bac5ac-c5f7-4842-ac25-9abd1b1e2851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:36.811577 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bac5ac-c5f7-4842-ac25-9abd1b1e2851 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.811874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bac5ac-c5f7-4842-ac25-9abd1b1e2851 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.812170 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bac5ac-c5f7-4842-ac25-9abd1b1e2851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.822538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bac5ac-c5f7-4842-ac25-9abd1b1e2851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:36.824493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bac5ac-c5f7-4842-ac25-9abd1b1e2851 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:36.835708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1036/2104] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:01:36.854238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bac5ac-c5f7-4842-ac25-9abd1b1e2851 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:36.856300 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c3bac5ac-c5f7-4842-ac25-9abd1b1e2851 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:36.876632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bac5ac-c5f7-4842-ac25-9abd1b1e2851 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:36.882927 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e1e33-622f-439d-b95e-dcd454e6eeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:36.883735 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e1e33-622f-439d-b95e-dcd454e6eeff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.884341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e1e33-622f-439d-b95e-dcd454e6eeff admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.884609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e1e33-622f-439d-b95e-dcd454e6eeff admin admin] PATH_INFO: `/v3/projects/f768b4d35c1b4713b58e3d582adcc93f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.886273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e1e33-622f-439d-b95e-dcd454e6eeff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f768b4d35c1b4713b58e3d582adcc93f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:36.888565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e1e33-622f-439d-b95e-dcd454e6eeff admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:36.930578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1069/2105] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:36 2022] PUT /identity/v3/projects/d572b7ff07cd47f18a2c2db1b9877804/users/c8b2be0d1e4b4456af201cf36eaf066b/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:36.936360 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db4455-0991-440e-bb96-9758cdf4cd44 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:37.006711 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db4455-0991-440e-bb96-9758cdf4cd44 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:37.011462 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb31af-a10a-498e-8df8-ed64f9e56e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:37.012095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb31af-a10a-498e-8df8-ed64f9e56e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.012371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb31af-a10a-498e-8df8-ed64f9e56e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.012641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb31af-a10a-498e-8df8-ed64f9e56e4a admin admin] PATH_INFO: `/v3/users/cf3eb99c308142388e527733ec12516a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.013949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb31af-a10a-498e-8df8-ed64f9e56e4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf3eb99c308142388e527733ec12516a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:37.015935 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb31af-a10a-498e-8df8-ed64f9e56e4a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:37.050950 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-811e1e33-622f-439d-b95e-dcd454e6eeff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f768b4d35c1b4713b58e3d582adcc93f', 'request_id': 'req-811e1e33-622f-439d-b95e-dcd454e6eeff'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:37.056546 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-811e1e33-622f-439d-b95e-dcd454e6eeff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f768b4d35c1b4713b58e3d582adcc93f', 'request_id': 'req-811e1e33-622f-439d-b95e-dcd454e6eeff'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:37.063513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-811e1e33-622f-439d-b95e-dcd454e6eeff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f768b4d35c1b4713b58e3d582adcc93f', 'request_id': 'req-811e1e33-622f-439d-b95e-dcd454e6eeff'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:37.151912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-811e1e33-622f-439d-b95e-dcd454e6eeff admin admin] The token cache is being invalidate because project f768b4d35c1b4713b58e3d582adcc93f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:37.152202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-811e1e33-622f-439d-b95e-dcd454e6eeff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:37.152910 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-811e1e33-622f-439d-b95e-dcd454e6eeff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:37.154656 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1037/2106] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:36 2022] DELETE /identity/v3/projects/f768b4d35c1b4713b58e3d582adcc93f => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:37.158800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef8747-6055-4f87-b5dd-ac719de73c33 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.158800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef8747-6055-4f87-b5dd-ac719de73c33 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.158800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef8747-6055-4f87-b5dd-ac719de73c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.188625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ddef8747-6055-4f87-b5dd-ac719de73c33 None None] MFA Rules not processed for user `c8b2be0d1e4b4456af201cf36eaf066b`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:37.189902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ddef8747-6055-4f87-b5dd-ac719de73c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:37.244371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5ebb31af-a10a-498e-8df8-ed64f9e56e4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf3eb99c308142388e527733ec12516a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:37.259511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1038/2107] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:37.260801 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d467419-a6af-4245-9687-f540eec3aec7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.261082 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d467419-a6af-4245-9687-f540eec3aec7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.261351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d467419-a6af-4245-9687-f540eec3aec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.282345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1d467419-a6af-4245-9687-f540eec3aec7 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:37.283608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1d467419-a6af-4245-9687-f540eec3aec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:37.317074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5ebb31af-a10a-498e-8df8-ed64f9e56e4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf3eb99c308142388e527733ec12516a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:37.317747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5ebb31af-a10a-498e-8df8-ed64f9e56e4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf3eb99c308142388e527733ec12516a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:37.334979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5ebb31af-a10a-498e-8df8-ed64f9e56e4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf3eb99c308142388e527733ec12516a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:37.348014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5ebb31af-a10a-498e-8df8-ed64f9e56e4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf3eb99c308142388e527733ec12516a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:37.348394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1039/2108] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:37.349216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d467419-a6af-4245-9687-f540eec3aec7 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:37.350567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1d467419-a6af-4245-9687-f540eec3aec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:37.362976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d467419-a6af-4245-9687-f540eec3aec7 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:37.363394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1070/2109] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:36 2022] DELETE /identity/v3/users/cf3eb99c308142388e527733ec12516a => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:37.366687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cf5fc-3844-43aa-94e4-50b4522ec9a4 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.366687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cf5fc-3844-43aa-94e4-50b4522ec9a4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.366687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cf5fc-3844-43aa-94e4-50b4522ec9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.386434 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ca3cf5fc-3844-43aa-94e4-50b4522ec9a4 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:37.387816 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ca3cf5fc-3844-43aa-94e4-50b4522ec9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:37.394551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:37.425509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfeaa67-7e69-410a-9cda-082c2123ee5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:37.431213 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfeaa67-7e69-410a-9cda-082c2123ee5d admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.431213 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfeaa67-7e69-410a-9cda-082c2123ee5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.431213 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfeaa67-7e69-410a-9cda-082c2123ee5d admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.431213 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfeaa67-7e69-410a-9cda-082c2123ee5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:37.431213 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfeaa67-7e69-410a-9cda-082c2123ee5d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:37.453151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1071/2110] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:37.454405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5bb0f-7393-46a9-b51f-15e60288e295 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.454690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5bb0f-7393-46a9-b51f-15e60288e295 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.454991 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a5bb0f-7393-46a9-b51f-15e60288e295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.462271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:37.475533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f4a5bb0f-7393-46a9-b51f-15e60288e295 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:37.476765 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f4a5bb0f-7393-46a9-b51f-15e60288e295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:37.487015 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1072/2111] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:37.487992 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5bb0f-7393-46a9-b51f-15e60288e295 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:37.498178 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a5bb0f-7393-46a9-b51f-15e60288e295 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:37.503412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ad46d-47e4-4c04-8f9d-65db8b22d304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:37.504105 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ad46d-47e4-4c04-8f9d-65db8b22d304 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.504605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ad46d-47e4-4c04-8f9d-65db8b22d304 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.504876 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ad46d-47e4-4c04-8f9d-65db8b22d304 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.505523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ad46d-47e4-4c04-8f9d-65db8b22d304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:37.505830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ad46d-47e4-4c04-8f9d-65db8b22d304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:37.507575 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ad46d-47e4-4c04-8f9d-65db8b22d304 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:37.521765 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1040/2112] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:37 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:37.521765 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfeaa67-7e69-410a-9cda-082c2123ee5d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:37.523175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1073/2113] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:37.525080 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ad46d-47e4-4c04-8f9d-65db8b22d304 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:37.527261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3dfeaa67-7e69-410a-9cda-082c2123ee5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:37.535564 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ad46d-47e4-4c04-8f9d-65db8b22d304 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:37.589097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfeaa67-7e69-410a-9cda-082c2123ee5d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:37.592773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b25c5-d34d-4bcd-8e0a-53632b219e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:37.593130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7812be-2f52-4737-9235-4c5a7bebe737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:37.593382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b25c5-d34d-4bcd-8e0a-53632b219e06 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.593661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b25c5-d34d-4bcd-8e0a-53632b219e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.593768 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7812be-2f52-4737-9235-4c5a7bebe737 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.593929 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b25c5-d34d-4bcd-8e0a-53632b219e06 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.594161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7812be-2f52-4737-9235-4c5a7bebe737 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.594311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7812be-2f52-4737-9235-4c5a7bebe737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.594594 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b25c5-d34d-4bcd-8e0a-53632b219e06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:37.596543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b25c5-d34d-4bcd-8e0a-53632b219e06 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:37.603966 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7812be-2f52-4737-9235-4c5a7bebe737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:37.605428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7812be-2f52-4737-9235-4c5a7bebe737 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:37.619895 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1041/2114] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:37.619895 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7812be-2f52-4737-9235-4c5a7bebe737 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:37.627367 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7812be-2f52-4737-9235-4c5a7bebe737 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:37.632954 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6901125-e132-4bcf-b97a-ee455954dd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:37.633708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6901125-e132-4bcf-b97a-ee455954dd9b admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.635871 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6901125-e132-4bcf-b97a-ee455954dd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.635871 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6901125-e132-4bcf-b97a-ee455954dd9b admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.635871 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6901125-e132-4bcf-b97a-ee455954dd9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:37.636950 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6901125-e132-4bcf-b97a-ee455954dd9b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:37.700975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1074/2115] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:37 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:37.704536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b25c5-d34d-4bcd-8e0a-53632b219e06 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:37.715313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b25c5-d34d-4bcd-8e0a-53632b219e06 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:37.720477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a0a38-6592-465c-ae86-9d93b0fb9299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:37.721109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a0a38-6592-465c-ae86-9d93b0fb9299 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.721371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a0a38-6592-465c-ae86-9d93b0fb9299 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.721673 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a0a38-6592-465c-ae86-9d93b0fb9299 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.722316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a0a38-6592-465c-ae86-9d93b0fb9299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:37.724042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a0a38-6592-465c-ae86-9d93b0fb9299 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:37.738079 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1075/2116] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:37.743040 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a0a38-6592-465c-ae86-9d93b0fb9299 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:37.743187 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1042/2117] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:37 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:37.747079 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6901125-e132-4bcf-b97a-ee455954dd9b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:37.754505 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a0a38-6592-465c-ae86-9d93b0fb9299 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:37.816013 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6901125-e132-4bcf-b97a-ee455954dd9b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:37.821076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3f444-d207-4fd6-99e7-1311f78c139d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:37.821691 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3f444-d207-4fd6-99e7-1311f78c139d admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.821973 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3f444-d207-4fd6-99e7-1311f78c139d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.822244 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3f444-d207-4fd6-99e7-1311f78c139d admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.822932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3f444-d207-4fd6-99e7-1311f78c139d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:37.824902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3f444-d207-4fd6-99e7-1311f78c139d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:37.830695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9402bb-df12-4f1a-92f2-25a11702ff70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:37.831370 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9402bb-df12-4f1a-92f2-25a11702ff70 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.832184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9402bb-df12-4f1a-92f2-25a11702ff70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.832184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9402bb-df12-4f1a-92f2-25a11702ff70 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.832678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9402bb-df12-4f1a-92f2-25a11702ff70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:37.835142 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9402bb-df12-4f1a-92f2-25a11702ff70 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:37.847917 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1076/2118] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:37.852785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9402bb-df12-4f1a-92f2-25a11702ff70 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:37.864588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9402bb-df12-4f1a-92f2-25a11702ff70 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:37.872415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59059bb-c638-46c3-a855-003fc3d0acdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:37.872415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59059bb-c638-46c3-a855-003fc3d0acdc admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.872415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59059bb-c638-46c3-a855-003fc3d0acdc admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.872415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59059bb-c638-46c3-a855-003fc3d0acdc admin admin] PATH_INFO: `/v3/projects/b10436bc5ae044d09bb6fc6e273af52e/users/af1d8ba605ac4c3992455eb47f04606a/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.890787 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1043/2119] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:37 2022] POST /identity/v3/users => generated 294 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:37.895368 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59059bb-c638-46c3-a855-003fc3d0acdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10436bc5ae044d09bb6fc6e273af52e, user_id=af1d8ba605ac4c3992455eb47f04606a, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:37.895494 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3f444-d207-4fd6-99e7-1311f78c139d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:37.896966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59059bb-c638-46c3-a855-003fc3d0acdc admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:37.907680 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3f444-d207-4fd6-99e7-1311f78c139d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:37.914932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3a17c-68b1-4fa7-8269-c3d36bf58cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:37.914932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3a17c-68b1-4fa7-8269-c3d36bf58cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.914932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3a17c-68b1-4fa7-8269-c3d36bf58cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.914932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3a17c-68b1-4fa7-8269-c3d36bf58cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.918790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3a17c-68b1-4fa7-8269-c3d36bf58cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:37.918790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3a17c-68b1-4fa7-8269-c3d36bf58cb9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:37.930240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1044/2120] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:37.933061 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3a17c-68b1-4fa7-8269-c3d36bf58cb9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:37.943761 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3a17c-68b1-4fa7-8269-c3d36bf58cb9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:37.948870 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ab6cb-6405-44e9-87a8-1edf9bd55135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:37.949478 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ab6cb-6405-44e9-87a8-1edf9bd55135 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.949761 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ab6cb-6405-44e9-87a8-1edf9bd55135 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.950036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ab6cb-6405-44e9-87a8-1edf9bd55135 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.950706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ab6cb-6405-44e9-87a8-1edf9bd55135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:37.952390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ab6cb-6405-44e9-87a8-1edf9bd55135 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:37.957874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:37.968419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1045/2121] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:37.969897 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ab6cb-6405-44e9-87a8-1edf9bd55135 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:37.980739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ab6cb-6405-44e9-87a8-1edf9bd55135 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:37.988177 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4d6d7-1559-4386-a3cb-ed8d5e8380fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:37.988743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4d6d7-1559-4386-a3cb-ed8d5e8380fe admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.989148 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4d6d7-1559-4386-a3cb-ed8d5e8380fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.989452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4d6d7-1559-4386-a3cb-ed8d5e8380fe admin admin] PATH_INFO: `/v3/projects/a9448ca7c8234f74b24c8f6710a0ef5c/users/48bf6dab289545bb80182c8904c05d6c/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.009999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4d6d7-1559-4386-a3cb-ed8d5e8380fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9448ca7c8234f74b24c8f6710a0ef5c, user_id=48bf6dab289545bb80182c8904c05d6c, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:38.012919 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4d6d7-1559-4386-a3cb-ed8d5e8380fe admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:38.020088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:38.030608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1077/2122] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:37 2022] PUT /identity/v3/projects/b10436bc5ae044d09bb6fc6e273af52e/users/af1d8ba605ac4c3992455eb47f04606a/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:38.040049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df6839-786f-4a1b-bb76-26f61a2dee1b None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.040332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df6839-786f-4a1b-bb76-26f61a2dee1b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.040606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df6839-786f-4a1b-bb76-26f61a2dee1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.073688 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-51df6839-786f-4a1b-bb76-26f61a2dee1b None None] MFA Rules not processed for user `af1d8ba605ac4c3992455eb47f04606a`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:38.076318 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-51df6839-786f-4a1b-bb76-26f61a2dee1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:38.146754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1078/2123] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:38.151813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df6839-786f-4a1b-bb76-26f61a2dee1b None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:38.225518 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df6839-786f-4a1b-bb76-26f61a2dee1b None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:38.227472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1046/2124] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:37 2022] PUT /identity/v3/projects/a9448ca7c8234f74b24c8f6710a0ef5c/users/48bf6dab289545bb80182c8904c05d6c/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:38.236980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4d6d7-1559-4386-a3cb-ed8d5e8380fe admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:38.236980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-23b4d6d7-1559-4386-a3cb-ed8d5e8380fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:38.287632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2d15b-5ea3-41ec-8c80-b045a1d2f7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:38.288313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2d15b-5ea3-41ec-8c80-b045a1d2f7cb admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.288613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2d15b-5ea3-41ec-8c80-b045a1d2f7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.288904 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2d15b-5ea3-41ec-8c80-b045a1d2f7cb admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.289560 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2d15b-5ea3-41ec-8c80-b045a1d2f7cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:38.291653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2d15b-5ea3-41ec-8c80-b045a1d2f7cb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:38.302500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4d6d7-1559-4386-a3cb-ed8d5e8380fe admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:38.307476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48842006-6465-470d-a925-a7005679ccfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:38.308123 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48842006-6465-470d-a925-a7005679ccfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.308495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48842006-6465-470d-a925-a7005679ccfc admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.308680 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48842006-6465-470d-a925-a7005679ccfc admin admin] PATH_INFO: `/v3/projects/c10d9bc9c0724621b7be95f79852bf0c` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.309937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48842006-6465-470d-a925-a7005679ccfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c10d9bc9c0724621b7be95f79852bf0c)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:38.311884 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48842006-6465-470d-a925-a7005679ccfc admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:38.411825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1079/2125] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:38 2022] POST /identity/v3/projects => generated 385 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:38.413105 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a4e52-0c70-4e72-ada8-71f55f5a88cd None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.413396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a4e52-0c70-4e72-ada8-71f55f5a88cd None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.413820 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a4e52-0c70-4e72-ada8-71f55f5a88cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.432930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-920a4e52-0c70-4e72-ada8-71f55f5a88cd None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:38.434290 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-920a4e52-0c70-4e72-ada8-71f55f5a88cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:38.462641 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-48842006-6465-470d-a925-a7005679ccfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c10d9bc9c0724621b7be95f79852bf0c', 'request_id': 'req-48842006-6465-470d-a925-a7005679ccfc'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:38.468332 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-48842006-6465-470d-a925-a7005679ccfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c10d9bc9c0724621b7be95f79852bf0c', 'request_id': 'req-48842006-6465-470d-a925-a7005679ccfc'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:38.481522 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-48842006-6465-470d-a925-a7005679ccfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c10d9bc9c0724621b7be95f79852bf0c', 'request_id': 'req-48842006-6465-470d-a925-a7005679ccfc'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:38.507454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1080/2126] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:38.507454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a4e52-0c70-4e72-ada8-71f55f5a88cd None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:38.515063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a4e52-0c70-4e72-ada8-71f55f5a88cd None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:38.567933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-48842006-6465-470d-a925-a7005679ccfc admin admin] The token cache is being invalidate because project c10d9bc9c0724621b7be95f79852bf0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:38.568226 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-48842006-6465-470d-a925-a7005679ccfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:38.568870 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-48842006-6465-470d-a925-a7005679ccfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:38.570441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1047/2127] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:38 2022] DELETE /identity/v3/projects/c10d9bc9c0724621b7be95f79852bf0c => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:38.571257 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48842006-6465-470d-a925-a7005679ccfc admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:38.574837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-48842006-6465-470d-a925-a7005679ccfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:38.575941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d54bb4-c4b0-4125-b266-68932d08e124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:38.576560 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d54bb4-c4b0-4125-b266-68932d08e124 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.576842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d54bb4-c4b0-4125-b266-68932d08e124 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.577116 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d54bb4-c4b0-4125-b266-68932d08e124 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.577753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d54bb4-c4b0-4125-b266-68932d08e124 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:38.579813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d54bb4-c4b0-4125-b266-68932d08e124 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:38.633861 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48842006-6465-470d-a925-a7005679ccfc admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:38.637571 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f6b1c-63fe-493f-a827-ada89181db10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:38.638179 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f6b1c-63fe-493f-a827-ada89181db10 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.638458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f6b1c-63fe-493f-a827-ada89181db10 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.638732 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f6b1c-63fe-493f-a827-ada89181db10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.640931 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e76f6b1c-63fe-493f-a827-ada89181db10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:38.654459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f6b1c-63fe-493f-a827-ada89181db10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:38.655992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f6b1c-63fe-493f-a827-ada89181db10 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:38.666215 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1048/2128] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:38.667459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febb08f-3ff6-43d8-9618-0d7dfd56462c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.667739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febb08f-3ff6-43d8-9618-0d7dfd56462c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.668080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febb08f-3ff6-43d8-9618-0d7dfd56462c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.679665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1081/2129] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:38 2022] POST /identity/v3/users => generated 299 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:38.684071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d54bb4-c4b0-4125-b266-68932d08e124 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:38.685203 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:38.686165 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e0d54bb4-c4b0-4125-b266-68932d08e124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:38.695643 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-6febb08f-3ff6-43d8-9618-0d7dfd56462c None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:38.697677 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6febb08f-3ff6-43d8-9618-0d7dfd56462c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:38.701411 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d54bb4-c4b0-4125-b266-68932d08e124 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:38.706536 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1049/2130] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:38.706778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37bb20c-4263-4da7-9674-da91284bc9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:38.707462 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37bb20c-4263-4da7-9674-da91284bc9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.707742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37bb20c-4263-4da7-9674-da91284bc9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.708017 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37bb20c-4263-4da7-9674-da91284bc9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.708661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37bb20c-4263-4da7-9674-da91284bc9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:38.710326 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37bb20c-4263-4da7-9674-da91284bc9d3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:38.714083 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7ee9e-c4d2-4abc-9f8b-9e24e3ab6104 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.714360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7ee9e-c4d2-4abc-9f8b-9e24e3ab6104 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.714633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7ee9e-c4d2-4abc-9f8b-9e24e3ab6104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.722488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1082/2131] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:38.727828 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37bb20c-4263-4da7-9674-da91284bc9d3 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:38.738129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37bb20c-4263-4da7-9674-da91284bc9d3 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:38.739460 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-fcc7ee9e-c4d2-4abc-9f8b-9e24e3ab6104 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:38.741045 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fcc7ee9e-c4d2-4abc-9f8b-9e24e3ab6104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:38.743185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8102983-6781-44a0-80c3-acc789753efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:38.743842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8102983-6781-44a0-80c3-acc789753efa admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.745559 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8102983-6781-44a0-80c3-acc789753efa admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.745845 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8102983-6781-44a0-80c3-acc789753efa admin admin] PATH_INFO: `/v3/projects/74550323be7b4c75be7493a51818d9b1/users/f6a1ab8b17974e2b8187ecc2d0fb1ad9/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.751009 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1050/2132] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:38.756651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7ee9e-c4d2-4abc-9f8b-9e24e3ab6104 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:38.767084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7ee9e-c4d2-4abc-9f8b-9e24e3ab6104 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:38.768211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8102983-6781-44a0-80c3-acc789753efa admin admin] RBAC: Authorizing `identity:create_grant(project_id=74550323be7b4c75be7493a51818d9b1, user_id=f6a1ab8b17974e2b8187ecc2d0fb1ad9, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:38.770936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8102983-6781-44a0-80c3-acc789753efa admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:38.771701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146944d-133b-4950-b461-65ddece80996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:38.772683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146944d-133b-4950-b461-65ddece80996 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.772683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146944d-133b-4950-b461-65ddece80996 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.772912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146944d-133b-4950-b461-65ddece80996 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.773554 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146944d-133b-4950-b461-65ddece80996 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:38.773854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146944d-133b-4950-b461-65ddece80996 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:38.775727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146944d-133b-4950-b461-65ddece80996 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:38.794930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1051/2133] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:01:38.798720 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146944d-133b-4950-b461-65ddece80996 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:38.809160 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146944d-133b-4950-b461-65ddece80996 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:38.814810 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dc20c-b6a4-4b3b-9149-f06de32125fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:38.815533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:38.815671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dc20c-b6a4-4b3b-9149-f06de32125fc admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.815776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dc20c-b6a4-4b3b-9149-f06de32125fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.815973 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dc20c-b6a4-4b3b-9149-f06de32125fc admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.816601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9dc20c-b6a4-4b3b-9149-f06de32125fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:38.818662 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9dc20c-b6a4-4b3b-9149-f06de32125fc admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:38.884783 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1083/2134] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:38 2022] PUT /identity/v3/projects/74550323be7b4c75be7493a51818d9b1/users/f6a1ab8b17974e2b8187ecc2d0fb1ad9/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:38.888789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8102983-6781-44a0-80c3-acc789753efa admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:38.914538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1052/2135] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:38 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:38.919051 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dc20c-b6a4-4b3b-9149-f06de32125fc admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:38.953572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8102983-6781-44a0-80c3-acc789753efa admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:38.991238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dc20c-b6a4-4b3b-9149-f06de32125fc admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:38.996732 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b79e4-99c3-4312-9151-bfdbd6377c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:38.997404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b79e4-99c3-4312-9151-bfdbd6377c98 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.997678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b79e4-99c3-4312-9151-bfdbd6377c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.998115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b79e4-99c3-4312-9151-bfdbd6377c98 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.998593 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b79e4-99c3-4312-9151-bfdbd6377c98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:39.001095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b79e4-99c3-4312-9151-bfdbd6377c98 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:39.038157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607bc3e-86b4-471b-bdac-f8d05a3b9158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:39.038783 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607bc3e-86b4-471b-bdac-f8d05a3b9158 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.039129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607bc3e-86b4-471b-bdac-f8d05a3b9158 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.041501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607bc3e-86b4-471b-bdac-f8d05a3b9158 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.041501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607bc3e-86b4-471b-bdac-f8d05a3b9158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:39.041754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607bc3e-86b4-471b-bdac-f8d05a3b9158 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:39.055007 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1084/2136] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:39.056731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607bc3e-86b4-471b-bdac-f8d05a3b9158 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:39.060695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:39.067517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607bc3e-86b4-471b-bdac-f8d05a3b9158 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:39.072652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6407c-f49a-4b9e-a1ce-f401c9dc1838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:39.073313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6407c-f49a-4b9e-a1ce-f401c9dc1838 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.073601 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6407c-f49a-4b9e-a1ce-f401c9dc1838 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.073914 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6407c-f49a-4b9e-a1ce-f401c9dc1838 admin admin] PATH_INFO: `/v3/projects/74550323be7b4c75be7493a51818d9b1/users/f6a1ab8b17974e2b8187ecc2d0fb1ad9/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.076142 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6407c-f49a-4b9e-a1ce-f401c9dc1838 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74550323be7b4c75be7493a51818d9b1, user_id=f6a1ab8b17974e2b8187ecc2d0fb1ad9, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:39.079887 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6407c-f49a-4b9e-a1ce-f401c9dc1838 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:39.091362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1085/2137] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:39 2022] PUT /identity/v3/projects/74550323be7b4c75be7493a51818d9b1/users/f6a1ab8b17974e2b8187ecc2d0fb1ad9/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:39.099010 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1053/2138] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:38 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:39.106690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6407c-f49a-4b9e-a1ce-f401c9dc1838 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:39.112625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a90e36-666a-4ce3-bd8e-6efdcd760748 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.112998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a90e36-666a-4ce3-bd8e-6efdcd760748 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.113371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a90e36-666a-4ce3-bd8e-6efdcd760748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.123813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:39.146424 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c7a90e36-666a-4ce3-bd8e-6efdcd760748 None None] MFA Rules not processed for user `f6a1ab8b17974e2b8187ecc2d0fb1ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:39.147934 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c7a90e36-666a-4ce3-bd8e-6efdcd760748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:39.177631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6407c-f49a-4b9e-a1ce-f401c9dc1838 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:39.185518 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564929e2-644d-4e5b-bfde-ffe517a9306f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:39.186200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564929e2-644d-4e5b-bfde-ffe517a9306f admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.186486 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564929e2-644d-4e5b-bfde-ffe517a9306f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.186853 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564929e2-644d-4e5b-bfde-ffe517a9306f admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.189331 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564929e2-644d-4e5b-bfde-ffe517a9306f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:39.191469 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564929e2-644d-4e5b-bfde-ffe517a9306f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:39.208923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1086/2139] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:39.210651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564929e2-644d-4e5b-bfde-ffe517a9306f admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:39.224533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564929e2-644d-4e5b-bfde-ffe517a9306f admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:39.230589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840462cb-ffce-497c-b9c3-eb2f16eaab34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:39.231516 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840462cb-ffce-497c-b9c3-eb2f16eaab34 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.236141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840462cb-ffce-497c-b9c3-eb2f16eaab34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.236141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840462cb-ffce-497c-b9c3-eb2f16eaab34 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.236141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840462cb-ffce-497c-b9c3-eb2f16eaab34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:39.236141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840462cb-ffce-497c-b9c3-eb2f16eaab34 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:39.244875 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1054/2140] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:39.249444 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1087/2141] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:39.254627 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a90e36-666a-4ce3-bd8e-6efdcd760748 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:39.258746 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840462cb-ffce-497c-b9c3-eb2f16eaab34 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:39.260541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-840462cb-ffce-497c-b9c3-eb2f16eaab34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:39.269890 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a90e36-666a-4ce3-bd8e-6efdcd760748 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:39.276590 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473cc41-3bcd-42e4-bcb8-a89f89ac931b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:39.277415 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473cc41-3bcd-42e4-bcb8-a89f89ac931b admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.278444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473cc41-3bcd-42e4-bcb8-a89f89ac931b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.278444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473cc41-3bcd-42e4-bcb8-a89f89ac931b admin admin] PATH_INFO: `/v3/projects/2f63c1ffd86d4bd1aa5989c9bb6d850f/users/bfb363a951f149438ddcd4ab8b819429/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.297829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473cc41-3bcd-42e4-bcb8-a89f89ac931b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f63c1ffd86d4bd1aa5989c9bb6d850f, user_id=bfb363a951f149438ddcd4ab8b819429, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:39.302034 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473cc41-3bcd-42e4-bcb8-a89f89ac931b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:39.364206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:39.386365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840462cb-ffce-497c-b9c3-eb2f16eaab34 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:39.391534 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8b7de-28af-4adf-a317-78aacd042807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:39.392532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8b7de-28af-4adf-a317-78aacd042807 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.392969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8b7de-28af-4adf-a317-78aacd042807 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.393245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8b7de-28af-4adf-a317-78aacd042807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.395503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-80b8b7de-28af-4adf-a317-78aacd042807 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:39.440124 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1055/2142] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:39 2022] PUT /identity/v3/projects/2f63c1ffd86d4bd1aa5989c9bb6d850f/users/bfb363a951f149438ddcd4ab8b819429/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:39.452662 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e189286-17ae-4e71-b9d8-31d8867a4a79 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.454372 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e189286-17ae-4e71-b9d8-31d8867a4a79 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.454870 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e189286-17ae-4e71-b9d8-31d8867a4a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.475939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8b7de-28af-4adf-a317-78aacd042807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:39.477775 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8b7de-28af-4adf-a317-78aacd042807 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:39.490243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-3e189286-17ae-4e71-b9d8-31d8867a4a79 None None] MFA Rules not processed for user `bfb363a951f149438ddcd4ab8b819429`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:39.491267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3e189286-17ae-4e71-b9d8-31d8867a4a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:39.496422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1088/2143] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:01:39.593136 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1056/2144] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:39.604900 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8b7de-28af-4adf-a317-78aacd042807 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:39.606924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-80b8b7de-28af-4adf-a317-78aacd042807 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:39.620490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8b7de-28af-4adf-a317-78aacd042807 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:39.670447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:39.682139 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39984b-e9a3-4124-9945-87df5fda14aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:39.682799 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39984b-e9a3-4124-9945-87df5fda14aa service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.683121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39984b-e9a3-4124-9945-87df5fda14aa service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.683391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39984b-e9a3-4124-9945-87df5fda14aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.692936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39984b-e9a3-4124-9945-87df5fda14aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:39.694367 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39984b-e9a3-4124-9945-87df5fda14aa service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:39.704503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1089/2145] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:42.551244 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00a657-b56a-4cb9-bc8e-f01cf26c641e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:42.551841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00a657-b56a-4cb9-bc8e-f01cf26c641e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:42.551841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00a657-b56a-4cb9-bc8e-f01cf26c641e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:42.571912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ce00a657-b56a-4cb9-bc8e-f01cf26c641e None None] MFA Rules not processed for user `bfb363a951f149438ddcd4ab8b819429`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:42.573119 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ce00a657-b56a-4cb9-bc8e-f01cf26c641e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:42.580743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1057/2146] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:42.773284 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39984b-e9a3-4124-9945-87df5fda14aa service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:42.787788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39984b-e9a3-4124-9945-87df5fda14aa service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:42.792251 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f65d5-287b-4890-95b6-f05b78c83eac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:42.792874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f65d5-287b-4890-95b6-f05b78c83eac service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:42.793151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f65d5-287b-4890-95b6-f05b78c83eac service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:42.793581 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f65d5-287b-4890-95b6-f05b78c83eac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:42.795480 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-926f65d5-287b-4890-95b6-f05b78c83eac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:42.864003 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f65d5-287b-4890-95b6-f05b78c83eac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:42.865506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f65d5-287b-4890-95b6-f05b78c83eac service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:42.876306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1090/2147] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:44.935871 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00a657-b56a-4cb9-bc8e-f01cf26c641e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:44.948501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce00a657-b56a-4cb9-bc8e-f01cf26c641e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:44.954350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bae795-914f-4684-a9e3-390b3ede2148 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:44.955008 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bae795-914f-4684-a9e3-390b3ede2148 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:44.955316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bae795-914f-4684-a9e3-390b3ede2148 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:44.955648 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bae795-914f-4684-a9e3-390b3ede2148 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:44.961850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:45.030878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bae795-914f-4684-a9e3-390b3ede2148 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:45.033140 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bae795-914f-4684-a9e3-390b3ede2148 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:45.056365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1058/2148] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:45.714724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f872f-2848-40e4-a57b-f0cd1d01ddd8 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:45.715303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f872f-2848-40e4-a57b-f0cd1d01ddd8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:45.715303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f872f-2848-40e4-a57b-f0cd1d01ddd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:45.745191 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b61f872f-2848-40e4-a57b-f0cd1d01ddd8 None None] MFA Rules not processed for user `48bf6dab289545bb80182c8904c05d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:45.746423 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b61f872f-2848-40e4-a57b-f0cd1d01ddd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:45.831329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1091/2149] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:45.847053 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bae795-914f-4684-a9e3-390b3ede2148 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:45.850550 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-66bae795-914f-4684-a9e3-390b3ede2148 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:45.972556 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bae795-914f-4684-a9e3-390b3ede2148 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:45.981015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28b50e-1147-413c-92f6-f7e9c64263b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:45.981996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28b50e-1147-413c-92f6-f7e9c64263b9 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:45.982339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28b50e-1147-413c-92f6-f7e9c64263b9 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:45.982808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28b50e-1147-413c-92f6-f7e9c64263b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:45.996244 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28b50e-1147-413c-92f6-f7e9c64263b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:45.998086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28b50e-1147-413c-92f6-f7e9c64263b9 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:46.012981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1059/2150] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:48.847431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfbb6f-aa21-4844-aa04-5c09e69f6b34 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:48.847950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfbb6f-aa21-4844-aa04-5c09e69f6b34 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:48.847950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfbb6f-aa21-4844-aa04-5c09e69f6b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:48.870274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-31cfbb6f-aa21-4844-aa04-5c09e69f6b34 None None] MFA Rules not processed for user `48bf6dab289545bb80182c8904c05d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:48.872441 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-31cfbb6f-aa21-4844-aa04-5c09e69f6b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:48.888644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1092/2151] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:50.255835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28b50e-1147-413c-92f6-f7e9c64263b9 service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:50.351086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28b50e-1147-413c-92f6-f7e9c64263b9 service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:50.357343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd095b3-3e41-4fa4-aadb-e0cc6de11267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:50.358000 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd095b3-3e41-4fa4-aadb-e0cc6de11267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:50.358272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd095b3-3e41-4fa4-aadb-e0cc6de11267 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:50.358545 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd095b3-3e41-4fa4-aadb-e0cc6de11267 admin admin] PATH_INFO: `/v3/users/c8b2be0d1e4b4456af201cf36eaf066b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:50.360369 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd095b3-3e41-4fa4-aadb-e0cc6de11267 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8b2be0d1e4b4456af201cf36eaf066b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:50.362521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd095b3-3e41-4fa4-aadb-e0cc6de11267 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:50.581320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:50.596290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ffd095b3-3e41-4fa4-aadb-e0cc6de11267 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8b2be0d1e4b4456af201cf36eaf066b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:50.694792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ffd095b3-3e41-4fa4-aadb-e0cc6de11267 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8b2be0d1e4b4456af201cf36eaf066b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:50.712985 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ffd095b3-3e41-4fa4-aadb-e0cc6de11267 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8b2be0d1e4b4456af201cf36eaf066b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:50.714040 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ffd095b3-3e41-4fa4-aadb-e0cc6de11267 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8b2be0d1e4b4456af201cf36eaf066b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:50.723158 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ffd095b3-3e41-4fa4-aadb-e0cc6de11267 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8b2be0d1e4b4456af201cf36eaf066b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:50.731085 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1060/2152] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:50 2022] DELETE /identity/v3/users/c8b2be0d1e4b4456af201cf36eaf066b => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:50.739656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34918d-e9c0-4e4e-b95d-a7dba9671ea5 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:50.739656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34918d-e9c0-4e4e-b95d-a7dba9671ea5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:50.739656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34918d-e9c0-4e4e-b95d-a7dba9671ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:50.771579 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-4d34918d-e9c0-4e4e-b95d-a7dba9671ea5 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:50.773556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4d34918d-e9c0-4e4e-b95d-a7dba9671ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:50.880140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1093/2153] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:50.893584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd095b3-3e41-4fa4-aadb-e0cc6de11267 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:50.895515 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ffd095b3-3e41-4fa4-aadb-e0cc6de11267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:51.005433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd095b3-3e41-4fa4-aadb-e0cc6de11267 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:51.009175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090fe3f-88f8-422a-a80f-a3bb3c583c35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:51.009843 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090fe3f-88f8-422a-a80f-a3bb3c583c35 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:51.010120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090fe3f-88f8-422a-a80f-a3bb3c583c35 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:51.010387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b090fe3f-88f8-422a-a80f-a3bb3c583c35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:51.021795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090fe3f-88f8-422a-a80f-a3bb3c583c35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:51.023407 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b090fe3f-88f8-422a-a80f-a3bb3c583c35 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:51.035100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1061/2154] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:51.608866 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34918d-e9c0-4e4e-b95d-a7dba9671ea5 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:51.611287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4d34918d-e9c0-4e4e-b95d-a7dba9671ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:51.634438 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34918d-e9c0-4e4e-b95d-a7dba9671ea5 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:51.655608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32574e3b-9074-4bff-acec-c420c73637c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:51.656833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32574e3b-9074-4bff-acec-c420c73637c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:51.657677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32574e3b-9074-4bff-acec-c420c73637c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:51.658213 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32574e3b-9074-4bff-acec-c420c73637c7 admin admin] PATH_INFO: `/v3/projects/d572b7ff07cd47f18a2c2db1b9877804` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:51.659709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32574e3b-9074-4bff-acec-c420c73637c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d572b7ff07cd47f18a2c2db1b9877804)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:51.662670 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32574e3b-9074-4bff-acec-c420c73637c7 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:51.664939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:51.818036 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-32574e3b-9074-4bff-acec-c420c73637c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd572b7ff07cd47f18a2c2db1b9877804', 'request_id': 'req-32574e3b-9074-4bff-acec-c420c73637c7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:51.900532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-32574e3b-9074-4bff-acec-c420c73637c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd572b7ff07cd47f18a2c2db1b9877804', 'request_id': 'req-32574e3b-9074-4bff-acec-c420c73637c7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:51.908844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-32574e3b-9074-4bff-acec-c420c73637c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd572b7ff07cd47f18a2c2db1b9877804', 'request_id': 'req-32574e3b-9074-4bff-acec-c420c73637c7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:51.921501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-32574e3b-9074-4bff-acec-c420c73637c7 admin admin] The token cache is being invalidate because project d572b7ff07cd47f18a2c2db1b9877804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:51.922034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-32574e3b-9074-4bff-acec-c420c73637c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:51.922779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-32574e3b-9074-4bff-acec-c420c73637c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:51.924381 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1094/2155] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:51 2022] DELETE /identity/v3/projects/d572b7ff07cd47f18a2c2db1b9877804 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:51.928562 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090fe3f-88f8-422a-a80f-a3bb3c583c35 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:51.940203 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b090fe3f-88f8-422a-a80f-a3bb3c583c35 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:52.035418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6e779-77a4-4fa8-9088-d35e87d25253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:52.036127 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6e779-77a4-4fa8-9088-d35e87d25253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:52.036624 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6e779-77a4-4fa8-9088-d35e87d25253 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:52.037127 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6e779-77a4-4fa8-9088-d35e87d25253 admin admin] PATH_INFO: `/v3/users/af1d8ba605ac4c3992455eb47f04606a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:52.038306 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6e779-77a4-4fa8-9088-d35e87d25253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1d8ba605ac4c3992455eb47f04606a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:52.040300 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6e779-77a4-4fa8-9088-d35e87d25253 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:52.298531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-16e6e779-77a4-4fa8-9088-d35e87d25253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1d8ba605ac4c3992455eb47f04606a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:52.303795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:52.758056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-16e6e779-77a4-4fa8-9088-d35e87d25253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1d8ba605ac4c3992455eb47f04606a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:52.781217 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-16e6e779-77a4-4fa8-9088-d35e87d25253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1d8ba605ac4c3992455eb47f04606a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:52.782547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-16e6e779-77a4-4fa8-9088-d35e87d25253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1d8ba605ac4c3992455eb47f04606a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:52.790278 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-16e6e779-77a4-4fa8-9088-d35e87d25253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1d8ba605ac4c3992455eb47f04606a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:52.796168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1062/2156] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:51 2022] DELETE /identity/v3/users/af1d8ba605ac4c3992455eb47f04606a => generated 0 bytes in 868 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:53.412539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32574e3b-9074-4bff-acec-c420c73637c7 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:53.413425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-32574e3b-9074-4bff-acec-c420c73637c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:53.426820 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32574e3b-9074-4bff-acec-c420c73637c7 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:53.493991 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30b3b0-ca23-4d10-91df-4b69bb103822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:53.494859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30b3b0-ca23-4d10-91df-4b69bb103822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:53.495225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30b3b0-ca23-4d10-91df-4b69bb103822 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:53.495508 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30b3b0-ca23-4d10-91df-4b69bb103822 admin admin] PATH_INFO: `/v3/projects/b10436bc5ae044d09bb6fc6e273af52e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:53.496745 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30b3b0-ca23-4d10-91df-4b69bb103822 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10436bc5ae044d09bb6fc6e273af52e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:53.498926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30b3b0-ca23-4d10-91df-4b69bb103822 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:53.635374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd30b3b0-ca23-4d10-91df-4b69bb103822 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10436bc5ae044d09bb6fc6e273af52e', 'request_id': 'req-cd30b3b0-ca23-4d10-91df-4b69bb103822'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:53.677038 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd30b3b0-ca23-4d10-91df-4b69bb103822 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10436bc5ae044d09bb6fc6e273af52e', 'request_id': 'req-cd30b3b0-ca23-4d10-91df-4b69bb103822'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:53.685810 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd30b3b0-ca23-4d10-91df-4b69bb103822 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10436bc5ae044d09bb6fc6e273af52e', 'request_id': 'req-cd30b3b0-ca23-4d10-91df-4b69bb103822'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:53.698273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd30b3b0-ca23-4d10-91df-4b69bb103822 admin admin] The token cache is being invalidate because project b10436bc5ae044d09bb6fc6e273af52e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:53.701521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd30b3b0-ca23-4d10-91df-4b69bb103822 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:53.701521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd30b3b0-ca23-4d10-91df-4b69bb103822 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:53.701521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1095/2157] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:53 2022] DELETE /identity/v3/projects/b10436bc5ae044d09bb6fc6e273af52e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:53.705316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6e779-77a4-4fa8-9088-d35e87d25253 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:53.706806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-16e6e779-77a4-4fa8-9088-d35e87d25253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:53.719976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6e779-77a4-4fa8-9088-d35e87d25253 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:53.785859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24ed6f-4cd5-4772-82cd-44d38747b6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:53.791146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24ed6f-4cd5-4772-82cd-44d38747b6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:53.791146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24ed6f-4cd5-4772-82cd-44d38747b6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:53.791146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24ed6f-4cd5-4772-82cd-44d38747b6a9 admin admin] PATH_INFO: `/v3/users/f6a1ab8b17974e2b8187ecc2d0fb1ad9` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:53.791146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24ed6f-4cd5-4772-82cd-44d38747b6a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6a1ab8b17974e2b8187ecc2d0fb1ad9)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:53.791146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24ed6f-4cd5-4772-82cd-44d38747b6a9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:53.986517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5e24ed6f-4cd5-4772-82cd-44d38747b6a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6a1ab8b17974e2b8187ecc2d0fb1ad9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:54.061606 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5e24ed6f-4cd5-4772-82cd-44d38747b6a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6a1ab8b17974e2b8187ecc2d0fb1ad9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:54.074807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5e24ed6f-4cd5-4772-82cd-44d38747b6a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6a1ab8b17974e2b8187ecc2d0fb1ad9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:54.075665 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5e24ed6f-4cd5-4772-82cd-44d38747b6a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6a1ab8b17974e2b8187ecc2d0fb1ad9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:54.084790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5e24ed6f-4cd5-4772-82cd-44d38747b6a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6a1ab8b17974e2b8187ecc2d0fb1ad9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:54.090938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1063/2158] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:53 2022] DELETE /identity/v3/users/f6a1ab8b17974e2b8187ecc2d0fb1ad9 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:54.636532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30b3b0-ca23-4d10-91df-4b69bb103822 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:54.636532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-cd30b3b0-ca23-4d10-91df-4b69bb103822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:54.652855 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30b3b0-ca23-4d10-91df-4b69bb103822 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:54.675918 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:54.719974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771de9ae-9ece-46fe-aed3-95bfebea7873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:54.720632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771de9ae-9ece-46fe-aed3-95bfebea7873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.721102 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771de9ae-9ece-46fe-aed3-95bfebea7873 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.721390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771de9ae-9ece-46fe-aed3-95bfebea7873 admin admin] PATH_INFO: `/v3/projects/74550323be7b4c75be7493a51818d9b1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.722515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771de9ae-9ece-46fe-aed3-95bfebea7873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74550323be7b4c75be7493a51818d9b1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:54.724516 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771de9ae-9ece-46fe-aed3-95bfebea7873 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:54.851803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-771de9ae-9ece-46fe-aed3-95bfebea7873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74550323be7b4c75be7493a51818d9b1', 'request_id': 'req-771de9ae-9ece-46fe-aed3-95bfebea7873'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:54.910566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-771de9ae-9ece-46fe-aed3-95bfebea7873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74550323be7b4c75be7493a51818d9b1', 'request_id': 'req-771de9ae-9ece-46fe-aed3-95bfebea7873'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:54.917319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-771de9ae-9ece-46fe-aed3-95bfebea7873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74550323be7b4c75be7493a51818d9b1', 'request_id': 'req-771de9ae-9ece-46fe-aed3-95bfebea7873'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:54.929623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-771de9ae-9ece-46fe-aed3-95bfebea7873 admin admin] The token cache is being invalidate because project 74550323be7b4c75be7493a51818d9b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:54.930013 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-771de9ae-9ece-46fe-aed3-95bfebea7873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:54.930776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-771de9ae-9ece-46fe-aed3-95bfebea7873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:54.937221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1096/2159] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:54 2022] DELETE /identity/v3/projects/74550323be7b4c75be7493a51818d9b1 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:54.938569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c458d20-660a-402b-8e0a-a4fd68d73344 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.938569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c458d20-660a-402b-8e0a-a4fd68d73344 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.938569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c458d20-660a-402b-8e0a-a4fd68d73344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.959585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4c458d20-660a-402b-8e0a-a4fd68d73344 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:54.960787 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4c458d20-660a-402b-8e0a-a4fd68d73344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:55.006828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:55.022216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1064/2160] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:55.031542 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c2fff-fdf2-45e2-b84c-89231722a08d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:55.031826 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c2fff-fdf2-45e2-b84c-89231722a08d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:55.032102 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c2fff-fdf2-45e2-b84c-89231722a08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:55.053278 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-191c2fff-fdf2-45e2-b84c-89231722a08d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:55.054458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-191c2fff-fdf2-45e2-b84c-89231722a08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:55.062545 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1097/2161] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:55.067716 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c458d20-660a-402b-8e0a-a4fd68d73344 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:55.077658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c458d20-660a-402b-8e0a-a4fd68d73344 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:55.082511 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278f7ed-c26e-4dd6-b68a-a3f3f23d658c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:55.083179 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278f7ed-c26e-4dd6-b68a-a3f3f23d658c admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:55.083466 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278f7ed-c26e-4dd6-b68a-a3f3f23d658c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:55.083740 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278f7ed-c26e-4dd6-b68a-a3f3f23d658c admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:55.084394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278f7ed-c26e-4dd6-b68a-a3f3f23d658c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:55.084702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278f7ed-c26e-4dd6-b68a-a3f3f23d658c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:55.086341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278f7ed-c26e-4dd6-b68a-a3f3f23d658c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:55.099228 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1065/2162] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:55.103437 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c2fff-fdf2-45e2-b84c-89231722a08d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:55.113229 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c2fff-fdf2-45e2-b84c-89231722a08d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:55.118014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4398388-dc99-4025-8d93-5013821e4f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:55.118603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4398388-dc99-4025-8d93-5013821e4f2a admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:55.118883 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4398388-dc99-4025-8d93-5013821e4f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:55.119185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4398388-dc99-4025-8d93-5013821e4f2a admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:55.119803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4398388-dc99-4025-8d93-5013821e4f2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:55.121775 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4398388-dc99-4025-8d93-5013821e4f2a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:55.232860 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1098/2163] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:55 2022] POST /identity/v3/projects => generated 359 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:55.237707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278f7ed-c26e-4dd6-b68a-a3f3f23d658c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:55.247971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278f7ed-c26e-4dd6-b68a-a3f3f23d658c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:55.308956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb806a-144a-49a7-92af-018f72c96532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:55.309479 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb806a-144a-49a7-92af-018f72c96532 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:55.309821 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb806a-144a-49a7-92af-018f72c96532 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:55.310050 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb806a-144a-49a7-92af-018f72c96532 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:55.310679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb806a-144a-49a7-92af-018f72c96532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:55.312639 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb806a-144a-49a7-92af-018f72c96532 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:55.449325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1066/2164] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:55 2022] POST /identity/v3/users => generated 280 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:55.453648 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4398388-dc99-4025-8d93-5013821e4f2a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:55.465288 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4398388-dc99-4025-8d93-5013821e4f2a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:55.476486 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6d4c7-3eb0-445e-88bf-f8b7875f29e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:55.476486 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6d4c7-3eb0-445e-88bf-f8b7875f29e5 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:55.483347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6d4c7-3eb0-445e-88bf-f8b7875f29e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:55.483839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6d4c7-3eb0-445e-88bf-f8b7875f29e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:55.484968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6d4c7-3eb0-445e-88bf-f8b7875f29e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:55.491681 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6d4c7-3eb0-445e-88bf-f8b7875f29e5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:55.509333 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1099/2165] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:55.510303 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb806a-144a-49a7-92af-018f72c96532 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:55.521103 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb806a-144a-49a7-92af-018f72c96532 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:55.527186 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cb467-8245-41f4-a6d7-4c57766b2c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:55.527886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cb467-8245-41f4-a6d7-4c57766b2c99 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:55.528153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cb467-8245-41f4-a6d7-4c57766b2c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:55.528430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cb467-8245-41f4-a6d7-4c57766b2c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:55.529142 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370cb467-8245-41f4-a6d7-4c57766b2c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:55.530737 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370cb467-8245-41f4-a6d7-4c57766b2c99 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:55.549829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1067/2166] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:55.553554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6d4c7-3eb0-445e-88bf-f8b7875f29e5 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:55.564578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6d4c7-3eb0-445e-88bf-f8b7875f29e5 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:55.573159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2b3d7-ce3b-4816-9834-9cd89abd4fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:55.573159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2b3d7-ce3b-4816-9834-9cd89abd4fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:55.573159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2b3d7-ce3b-4816-9834-9cd89abd4fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:55.573159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2b3d7-ce3b-4816-9834-9cd89abd4fdf admin admin] PATH_INFO: `/v3/projects/4666b3c98e7b4830a798fb30631140eb/users/345303d8b254401189917dac0eaa2905/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:55.586429 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2b3d7-ce3b-4816-9834-9cd89abd4fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4666b3c98e7b4830a798fb30631140eb, user_id=345303d8b254401189917dac0eaa2905, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:55.589122 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2b3d7-ce3b-4816-9834-9cd89abd4fdf admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:55.658838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1100/2167] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:55 2022] PUT /identity/v3/projects/4666b3c98e7b4830a798fb30631140eb/users/345303d8b254401189917dac0eaa2905/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:55.669477 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff5049-c5ac-4ab9-a50e-f2aa42f47213 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:55.669843 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff5049-c5ac-4ab9-a50e-f2aa42f47213 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:55.670513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff5049-c5ac-4ab9-a50e-f2aa42f47213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:55.710977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-2aff5049-c5ac-4ab9-a50e-f2aa42f47213 None None] MFA Rules not processed for user `345303d8b254401189917dac0eaa2905`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:55.712237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2aff5049-c5ac-4ab9-a50e-f2aa42f47213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:55.808567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1068/2168] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:01:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:55.823893 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2b3d7-ce3b-4816-9834-9cd89abd4fdf admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:55.826402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-25b2b3d7-ce3b-4816-9834-9cd89abd4fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:55.946755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:55.953124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2b3d7-ce3b-4816-9834-9cd89abd4fdf admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:55.958010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f41f7-8cd8-4743-91e9-e6fed86278e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:55.958856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f41f7-8cd8-4743-91e9-e6fed86278e7 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:55.959336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f41f7-8cd8-4743-91e9-e6fed86278e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:55.959727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f41f7-8cd8-4743-91e9-e6fed86278e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:55.970840 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f41f7-8cd8-4743-91e9-e6fed86278e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:55.972343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f41f7-8cd8-4743-91e9-e6fed86278e7 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:55.984929 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1101/2169] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:55 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:02.218851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff5049-c5ac-4ab9-a50e-f2aa42f47213 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:02.220315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2aff5049-c5ac-4ab9-a50e-f2aa42f47213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:02.269185 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:02.287791 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff5049-c5ac-4ab9-a50e-f2aa42f47213 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:02.292350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13754a2e-b8f8-47e7-9236-6032ad453c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:02.292976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13754a2e-b8f8-47e7-9236-6032ad453c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:02.293256 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13754a2e-b8f8-47e7-9236-6032ad453c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:02.293530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13754a2e-b8f8-47e7-9236-6032ad453c83 admin admin] PATH_INFO: `/v3/users/1187acb9ae484cbbb0451bdb6d8ff269` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:02.294650 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13754a2e-b8f8-47e7-9236-6032ad453c83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1187acb9ae484cbbb0451bdb6d8ff269)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:02.296617 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13754a2e-b8f8-47e7-9236-6032ad453c83 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:02.566213 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-13754a2e-b8f8-47e7-9236-6032ad453c83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1187acb9ae484cbbb0451bdb6d8ff269'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:02.648614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-13754a2e-b8f8-47e7-9236-6032ad453c83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1187acb9ae484cbbb0451bdb6d8ff269'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:02.662919 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-13754a2e-b8f8-47e7-9236-6032ad453c83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1187acb9ae484cbbb0451bdb6d8ff269'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:02.663753 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-13754a2e-b8f8-47e7-9236-6032ad453c83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1187acb9ae484cbbb0451bdb6d8ff269'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:02.673153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-13754a2e-b8f8-47e7-9236-6032ad453c83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1187acb9ae484cbbb0451bdb6d8ff269'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:02.680345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1069/2170] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:02 2022] DELETE /identity/v3/users/1187acb9ae484cbbb0451bdb6d8ff269 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:02.684835 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599265f4-3327-419f-9e60-608a24024196 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:02.685312 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599265f4-3327-419f-9e60-608a24024196 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:02.685465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599265f4-3327-419f-9e60-608a24024196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:02.707817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-599265f4-3327-419f-9e60-608a24024196 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:02.708970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-599265f4-3327-419f-9e60-608a24024196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:02.770006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1102/2171] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:02.780095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13754a2e-b8f8-47e7-9236-6032ad453c83 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:02.781550 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-13754a2e-b8f8-47e7-9236-6032ad453c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:02.848829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13754a2e-b8f8-47e7-9236-6032ad453c83 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:02.853038 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6940f-fee7-4af0-b4f4-216ebc686c83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:02.853736 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6940f-fee7-4af0-b4f4-216ebc686c83 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:02.854031 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6940f-fee7-4af0-b4f4-216ebc686c83 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:02.854438 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6940f-fee7-4af0-b4f4-216ebc686c83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:02.864832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6940f-fee7-4af0-b4f4-216ebc686c83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:02.866270 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6940f-fee7-4af0-b4f4-216ebc686c83 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:02.876528 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1070/2172] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:03.497463 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599265f4-3327-419f-9e60-608a24024196 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:03.499227 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-599265f4-3327-419f-9e60-608a24024196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:03.520118 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599265f4-3327-419f-9e60-608a24024196 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:03.524878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb84fd-e443-4550-ba09-b8ff601ec95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:03.526655 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb84fd-e443-4550-ba09-b8ff601ec95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:03.527128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb84fd-e443-4550-ba09-b8ff601ec95d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:03.527648 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb84fd-e443-4550-ba09-b8ff601ec95d admin admin] PATH_INFO: `/v3/projects/4737f0a5e0164f309c591eba2a97ad0a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:03.529041 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb84fd-e443-4550-ba09-b8ff601ec95d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4737f0a5e0164f309c591eba2a97ad0a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:03.531083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb84fd-e443-4550-ba09-b8ff601ec95d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:03.646376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-65bb84fd-e443-4550-ba09-b8ff601ec95d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4737f0a5e0164f309c591eba2a97ad0a', 'request_id': 'req-65bb84fd-e443-4550-ba09-b8ff601ec95d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:03.723291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-65bb84fd-e443-4550-ba09-b8ff601ec95d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4737f0a5e0164f309c591eba2a97ad0a', 'request_id': 'req-65bb84fd-e443-4550-ba09-b8ff601ec95d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:03.732300 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-65bb84fd-e443-4550-ba09-b8ff601ec95d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4737f0a5e0164f309c591eba2a97ad0a', 'request_id': 'req-65bb84fd-e443-4550-ba09-b8ff601ec95d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:03.744688 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-65bb84fd-e443-4550-ba09-b8ff601ec95d admin admin] The token cache is being invalidate because project 4737f0a5e0164f309c591eba2a97ad0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:03.744907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-65bb84fd-e443-4550-ba09-b8ff601ec95d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:03.746518 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-65bb84fd-e443-4550-ba09-b8ff601ec95d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:03.747137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1103/2173] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:03 2022] DELETE /identity/v3/projects/4737f0a5e0164f309c591eba2a97ad0a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:03.758880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cf791-e8e4-4325-84b4-6bfa43b7cab1 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:03.759445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cf791-e8e4-4325-84b4-6bfa43b7cab1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:03.759571 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737cf791-e8e4-4325-84b4-6bfa43b7cab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:03.780649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-737cf791-e8e4-4325-84b4-6bfa43b7cab1 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:03.781878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-737cf791-e8e4-4325-84b4-6bfa43b7cab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:03.854301 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1071/2174] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:03.862668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93627c-ac76-4e0b-ad60-127b1e79b610 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:03.863351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93627c-ac76-4e0b-ad60-127b1e79b610 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:03.864005 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93627c-ac76-4e0b-ad60-127b1e79b610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:03.891094 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-8b93627c-ac76-4e0b-ad60-127b1e79b610 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:03.893172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8b93627c-ac76-4e0b-ad60-127b1e79b610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:03.906282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1104/2175] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:03.911404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cf791-e8e4-4325-84b4-6bfa43b7cab1 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:03.924555 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737cf791-e8e4-4325-84b4-6bfa43b7cab1 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:03.930127 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b096c-8808-44ed-8c3a-d2789e214e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:03.930468 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b096c-8808-44ed-8c3a-d2789e214e76 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:03.930748 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b096c-8808-44ed-8c3a-d2789e214e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:03.931069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b096c-8808-44ed-8c3a-d2789e214e76 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:03.931719 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b096c-8808-44ed-8c3a-d2789e214e76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:03.932021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b096c-8808-44ed-8c3a-d2789e214e76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:03.933707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b096c-8808-44ed-8c3a-d2789e214e76 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:03.935489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:03.958031 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1072/2176] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:02:03.958844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93627c-ac76-4e0b-ad60-127b1e79b610 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:03.969555 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93627c-ac76-4e0b-ad60-127b1e79b610 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:03.975349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1ca65-8adf-4fbb-a4b5-32a546826516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:03.976006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1ca65-8adf-4fbb-a4b5-32a546826516 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:03.976327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1ca65-8adf-4fbb-a4b5-32a546826516 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:03.976590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda1ca65-8adf-4fbb-a4b5-32a546826516 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:03.977280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1ca65-8adf-4fbb-a4b5-32a546826516 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:03.979202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda1ca65-8adf-4fbb-a4b5-32a546826516 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:04.084119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1105/2177] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:03 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:04.085322 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b096c-8808-44ed-8c3a-d2789e214e76 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:04.096366 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b096c-8808-44ed-8c3a-d2789e214e76 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:04.153574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642767a-f6a5-4413-a701-09b9b559b861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:04.154170 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642767a-f6a5-4413-a701-09b9b559b861 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:04.154417 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642767a-f6a5-4413-a701-09b9b559b861 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:04.154687 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642767a-f6a5-4413-a701-09b9b559b861 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:04.155340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642767a-f6a5-4413-a701-09b9b559b861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:04.157363 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642767a-f6a5-4413-a701-09b9b559b861 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:04.489166 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1073/2178] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:04 2022] POST /identity/v3/users => generated 287 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:04.489410 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1ca65-8adf-4fbb-a4b5-32a546826516 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:04.498309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda1ca65-8adf-4fbb-a4b5-32a546826516 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:04.503204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec334a-7403-4c1e-a29d-ad5423edca59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:04.504454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec334a-7403-4c1e-a29d-ad5423edca59 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:04.509513 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec334a-7403-4c1e-a29d-ad5423edca59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:04.509513 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec334a-7403-4c1e-a29d-ad5423edca59 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:04.509513 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec334a-7403-4c1e-a29d-ad5423edca59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:04.509513 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ec334a-7403-4c1e-a29d-ad5423edca59 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:04.519925 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1106/2179] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:04.523721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642767a-f6a5-4413-a701-09b9b559b861 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:04.535094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642767a-f6a5-4413-a701-09b9b559b861 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:04.540349 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9618ddc-7d8e-404a-9265-e25aa010798c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:04.540807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9618ddc-7d8e-404a-9265-e25aa010798c admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:04.541217 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9618ddc-7d8e-404a-9265-e25aa010798c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:04.541496 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9618ddc-7d8e-404a-9265-e25aa010798c admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:04.542166 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9618ddc-7d8e-404a-9265-e25aa010798c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:04.543790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9618ddc-7d8e-404a-9265-e25aa010798c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:04.555366 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1074/2180] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:04.559677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec334a-7403-4c1e-a29d-ad5423edca59 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:04.570321 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec334a-7403-4c1e-a29d-ad5423edca59 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:04.575440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf665860-92ec-48e7-9e19-c2c47aeb8430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:04.576104 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf665860-92ec-48e7-9e19-c2c47aeb8430 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:04.576387 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf665860-92ec-48e7-9e19-c2c47aeb8430 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:04.576664 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf665860-92ec-48e7-9e19-c2c47aeb8430 admin admin] PATH_INFO: `/v3/projects/b04c03ce7f6c4170ae6f1711a2918d5c/users/32c16912d9f9472a84dc88dbc13e1503/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:04.594557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf665860-92ec-48e7-9e19-c2c47aeb8430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04c03ce7f6c4170ae6f1711a2918d5c, user_id=32c16912d9f9472a84dc88dbc13e1503, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:04.598249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf665860-92ec-48e7-9e19-c2c47aeb8430 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:04.704642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1107/2181] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:04 2022] PUT /identity/v3/projects/b04c03ce7f6c4170ae6f1711a2918d5c/users/32c16912d9f9472a84dc88dbc13e1503/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:04.720050 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a38c9f-80b0-4c22-9486-5cc74c6848fb None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:04.720541 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a38c9f-80b0-4c22-9486-5cc74c6848fb None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:04.721515 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a38c9f-80b0-4c22-9486-5cc74c6848fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:04.762695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-64a38c9f-80b0-4c22-9486-5cc74c6848fb None None] MFA Rules not processed for user `32c16912d9f9472a84dc88dbc13e1503`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:04.763924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-64a38c9f-80b0-4c22-9486-5cc74c6848fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:04.841331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1075/2182] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:02:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:04.849744 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf665860-92ec-48e7-9e19-c2c47aeb8430 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:04.851614 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-bf665860-92ec-48e7-9e19-c2c47aeb8430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:04.863197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:04.918998 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf665860-92ec-48e7-9e19-c2c47aeb8430 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:04.922636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c417b6-5459-405f-acf2-fc4cd1a6bd10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:04.923271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c417b6-5459-405f-acf2-fc4cd1a6bd10 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:04.923588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c417b6-5459-405f-acf2-fc4cd1a6bd10 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:04.923864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c417b6-5459-405f-acf2-fc4cd1a6bd10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:04.932270 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c417b6-5459-405f-acf2-fc4cd1a6bd10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:04.933698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c417b6-5459-405f-acf2-fc4cd1a6bd10 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:04.943815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1108/2183] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:14.833799 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:15.089644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=22 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:17.368700 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118751d-77fb-4b01-824d-aa1b1a5aea1d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:17.369518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118751d-77fb-4b01-824d-aa1b1a5aea1d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:17.377148 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118751d-77fb-4b01-824d-aa1b1a5aea1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:17.401391 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0118751d-77fb-4b01-824d-aa1b1a5aea1d None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:17.402625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0118751d-77fb-4b01-824d-aa1b1a5aea1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:17.465891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1076/2184] 10.222.0.91 () {58 vars in 1009 bytes} [Thu Sep 8 14:02:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:17.525381 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c417b6-5459-405f-acf2-fc4cd1a6bd10 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:17.528663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-79c417b6-5459-405f-acf2-fc4cd1a6bd10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:17.624302 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c417b6-5459-405f-acf2-fc4cd1a6bd10 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:17.631414 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d27869-acdb-48e6-8729-968ba64aec74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:17.631714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d27869-acdb-48e6-8729-968ba64aec74 service swift] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:17.632001 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d27869-acdb-48e6-8729-968ba64aec74 service swift] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:17.632279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d27869-acdb-48e6-8729-968ba64aec74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:17.642296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d27869-acdb-48e6-8729-968ba64aec74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:17.643973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d27869-acdb-48e6-8729-968ba64aec74 service swift] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:17.661916 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1109/2185] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 14:02:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:02:18.927811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d06d8-bbbe-4474-9c3d-f10518f4acef None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:18.927811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d06d8-bbbe-4474-9c3d-f10518f4acef None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:18.927811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d06d8-bbbe-4474-9c3d-f10518f4acef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:18.927811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f85d06d8-bbbe-4474-9c3d-f10518f4acef None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:18.927811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f85d06d8-bbbe-4474-9c3d-f10518f4acef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:18.927811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1077/2186] 10.222.0.91 () {58 vars in 967 bytes} [Thu Sep 8 14:02:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:18.933532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d27869-acdb-48e6-8729-968ba64aec74 service swift] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:18.933532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d27869-acdb-48e6-8729-968ba64aec74 service swift] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:18.933532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ce5da-6981-4017-a926-9d2c6075e083 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:18.933532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ce5da-6981-4017-a926-9d2c6075e083 service swift] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:18.935325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ce5da-6981-4017-a926-9d2c6075e083 service swift] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:18.935325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ce5da-6981-4017-a926-9d2c6075e083 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:18.935325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ce5da-6981-4017-a926-9d2c6075e083 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:18.935325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ce5da-6981-4017-a926-9d2c6075e083 service swift] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:18.935325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1110/2187] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 14:02:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:02:19.523181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae072fe3-18bf-4e91-8279-7d25cbd361d9 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:19.523466 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae072fe3-18bf-4e91-8279-7d25cbd361d9 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:19.523736 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae072fe3-18bf-4e91-8279-7d25cbd361d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:19.545144 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ae072fe3-18bf-4e91-8279-7d25cbd361d9 None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:19.546394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ae072fe3-18bf-4e91-8279-7d25cbd361d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:19.554682 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1078/2188] 10.222.0.91 () {58 vars in 1009 bytes} [Thu Sep 8 14:02:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:19.566571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ce5da-6981-4017-a926-9d2c6075e083 service swift] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:19.578019 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ce5da-6981-4017-a926-9d2c6075e083 service swift] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:19.584076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf1551-1688-47aa-9486-7f3502cb774c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:19.584677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf1551-1688-47aa-9486-7f3502cb774c service swift] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:19.590534 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf1551-1688-47aa-9486-7f3502cb774c service swift] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:19.590534 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf1551-1688-47aa-9486-7f3502cb774c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:19.593787 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf1551-1688-47aa-9486-7f3502cb774c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:19.596145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf1551-1688-47aa-9486-7f3502cb774c service swift] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:19.605591 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1111/2189] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 14:02:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:02:20.482169 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22071d69-1ff0-4d15-87a6-1ade7b41229d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:20.482169 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22071d69-1ff0-4d15-87a6-1ade7b41229d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:20.482169 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22071d69-1ff0-4d15-87a6-1ade7b41229d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:20.509202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-22071d69-1ff0-4d15-87a6-1ade7b41229d None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:20.510413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-22071d69-1ff0-4d15-87a6-1ade7b41229d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:20.519478 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1079/2190] 10.222.0.91 () {58 vars in 967 bytes} [Thu Sep 8 14:02:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:20.532375 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf1551-1688-47aa-9486-7f3502cb774c service swift] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:20.543868 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf1551-1688-47aa-9486-7f3502cb774c service swift] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:20.549386 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fece07-aac2-4e7e-bac7-9f39eeb0969b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:20.550010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fece07-aac2-4e7e-bac7-9f39eeb0969b service swift] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:20.550297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fece07-aac2-4e7e-bac7-9f39eeb0969b service swift] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:20.550571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fece07-aac2-4e7e-bac7-9f39eeb0969b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:20.559538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fece07-aac2-4e7e-bac7-9f39eeb0969b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:20.561167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fece07-aac2-4e7e-bac7-9f39eeb0969b service swift] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:20.570371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1112/2191] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 14:02:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 14:02:21.544096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22071d69-1ff0-4d15-87a6-1ade7b41229d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:21.616371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22071d69-1ff0-4d15-87a6-1ade7b41229d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:21.621343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf391fdb-417f-4e09-ba65-3ef9fc02c00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:21.624026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf391fdb-417f-4e09-ba65-3ef9fc02c00c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:21.624026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf391fdb-417f-4e09-ba65-3ef9fc02c00c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:21.624026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf391fdb-417f-4e09-ba65-3ef9fc02c00c admin admin] PATH_INFO: `/v3/users/32c16912d9f9472a84dc88dbc13e1503` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:21.624026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf391fdb-417f-4e09-ba65-3ef9fc02c00c admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c16912d9f9472a84dc88dbc13e1503)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:21.625638 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf391fdb-417f-4e09-ba65-3ef9fc02c00c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:21.832093 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cf391fdb-417f-4e09-ba65-3ef9fc02c00c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c16912d9f9472a84dc88dbc13e1503'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:21.894422 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cf391fdb-417f-4e09-ba65-3ef9fc02c00c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c16912d9f9472a84dc88dbc13e1503'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:21.917354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cf391fdb-417f-4e09-ba65-3ef9fc02c00c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c16912d9f9472a84dc88dbc13e1503'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:21.917354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cf391fdb-417f-4e09-ba65-3ef9fc02c00c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c16912d9f9472a84dc88dbc13e1503'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:21.925053 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cf391fdb-417f-4e09-ba65-3ef9fc02c00c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c16912d9f9472a84dc88dbc13e1503'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:21.935343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1080/2192] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:21 2022] DELETE /identity/v3/users/32c16912d9f9472a84dc88dbc13e1503 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:21.940156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d502f40-4ba5-4aa2-8d07-63b950da3f14 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:21.940861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d502f40-4ba5-4aa2-8d07-63b950da3f14 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:21.941201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d502f40-4ba5-4aa2-8d07-63b950da3f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:21.978450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-6d502f40-4ba5-4aa2-8d07-63b950da3f14 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:21.980525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6d502f40-4ba5-4aa2-8d07-63b950da3f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:22.067818 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1113/2193] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:22.087146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf391fdb-417f-4e09-ba65-3ef9fc02c00c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:22.089696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-cf391fdb-417f-4e09-ba65-3ef9fc02c00c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:22.100028 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:22.175363 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf391fdb-417f-4e09-ba65-3ef9fc02c00c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:22.179209 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf21c0-ce40-49a3-a885-3c294e2d782c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:22.179795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf21c0-ce40-49a3-a885-3c294e2d782c service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:22.180214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf21c0-ce40-49a3-a885-3c294e2d782c service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:22.180494 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf21c0-ce40-49a3-a885-3c294e2d782c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:22.192544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf21c0-ce40-49a3-a885-3c294e2d782c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:22.194878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf21c0-ce40-49a3-a885-3c294e2d782c service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:22.207282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1081/2194] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:22.757400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d502f40-4ba5-4aa2-8d07-63b950da3f14 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:22.759503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6d502f40-4ba5-4aa2-8d07-63b950da3f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:22.778712 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d502f40-4ba5-4aa2-8d07-63b950da3f14 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:22.787014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea1235-3357-4ff1-9203-f5f2ea28d459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:22.788068 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea1235-3357-4ff1-9203-f5f2ea28d459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:22.788694 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea1235-3357-4ff1-9203-f5f2ea28d459 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:22.788968 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea1235-3357-4ff1-9203-f5f2ea28d459 admin admin] PATH_INFO: `/v3/projects/b04c03ce7f6c4170ae6f1711a2918d5c` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:22.793176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea1235-3357-4ff1-9203-f5f2ea28d459 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b04c03ce7f6c4170ae6f1711a2918d5c)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:22.797149 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea1235-3357-4ff1-9203-f5f2ea28d459 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:22.956884 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:22.960305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2bea1235-3357-4ff1-9203-f5f2ea28d459 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b04c03ce7f6c4170ae6f1711a2918d5c', 'request_id': 'req-2bea1235-3357-4ff1-9203-f5f2ea28d459'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.054679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2bea1235-3357-4ff1-9203-f5f2ea28d459 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b04c03ce7f6c4170ae6f1711a2918d5c', 'request_id': 'req-2bea1235-3357-4ff1-9203-f5f2ea28d459'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.062934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2bea1235-3357-4ff1-9203-f5f2ea28d459 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b04c03ce7f6c4170ae6f1711a2918d5c', 'request_id': 'req-2bea1235-3357-4ff1-9203-f5f2ea28d459'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.075259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2bea1235-3357-4ff1-9203-f5f2ea28d459 admin admin] The token cache is being invalidate because project b04c03ce7f6c4170ae6f1711a2918d5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:23.075706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2bea1235-3357-4ff1-9203-f5f2ea28d459 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.087572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2bea1235-3357-4ff1-9203-f5f2ea28d459 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.087572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1114/2195] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:22 2022] DELETE /identity/v3/projects/b04c03ce7f6c4170ae6f1711a2918d5c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:23.299607 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf21c0-ce40-49a3-a885-3c294e2d782c service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:23.301056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-aedf21c0-ce40-49a3-a885-3c294e2d782c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:23.317031 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf21c0-ce40-49a3-a885-3c294e2d782c service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:23.402112 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da241205-bcc8-42ec-90e4-46485e27215f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:23.402731 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da241205-bcc8-42ec-90e4-46485e27215f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:23.403041 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da241205-bcc8-42ec-90e4-46485e27215f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:23.403311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da241205-bcc8-42ec-90e4-46485e27215f admin admin] PATH_INFO: `/v3/users/bfb363a951f149438ddcd4ab8b819429` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:23.404438 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da241205-bcc8-42ec-90e4-46485e27215f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfb363a951f149438ddcd4ab8b819429)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:23.406337 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da241205-bcc8-42ec-90e4-46485e27215f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:23.612913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-da241205-bcc8-42ec-90e4-46485e27215f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfb363a951f149438ddcd4ab8b819429'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.707464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-da241205-bcc8-42ec-90e4-46485e27215f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfb363a951f149438ddcd4ab8b819429'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.723338 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-da241205-bcc8-42ec-90e4-46485e27215f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfb363a951f149438ddcd4ab8b819429'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.724834 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-da241205-bcc8-42ec-90e4-46485e27215f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfb363a951f149438ddcd4ab8b819429'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.734147 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-da241205-bcc8-42ec-90e4-46485e27215f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfb363a951f149438ddcd4ab8b819429'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.743265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1082/2196] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:23 2022] DELETE /identity/v3/users/bfb363a951f149438ddcd4ab8b819429 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:23.747783 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297460d-0c6b-43ce-8fd2-b3fef8089f3e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:23.748577 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297460d-0c6b-43ce-8fd2-b3fef8089f3e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:23.748946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297460d-0c6b-43ce-8fd2-b3fef8089f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:23.778282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-4297460d-0c6b-43ce-8fd2-b3fef8089f3e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:23.779567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4297460d-0c6b-43ce-8fd2-b3fef8089f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:23.851316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1115/2197] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:23.863211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da241205-bcc8-42ec-90e4-46485e27215f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:23.868654 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-da241205-bcc8-42ec-90e4-46485e27215f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:23.897628 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:23.934654 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da241205-bcc8-42ec-90e4-46485e27215f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:23.941571 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea5bad-68e2-42a0-82ce-60e9c362794f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:23.942312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea5bad-68e2-42a0-82ce-60e9c362794f service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:23.942688 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea5bad-68e2-42a0-82ce-60e9c362794f service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:23.943114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea5bad-68e2-42a0-82ce-60e9c362794f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:23.957961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea5bad-68e2-42a0-82ce-60e9c362794f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:23.959626 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea5bad-68e2-42a0-82ce-60e9c362794f service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:23.974350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1083/2198] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:24.318168 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297460d-0c6b-43ce-8fd2-b3fef8089f3e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.320256 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4297460d-0c6b-43ce-8fd2-b3fef8089f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:24.334317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297460d-0c6b-43ce-8fd2-b3fef8089f3e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.354045 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2a828-e09a-4cdc-8b77-60b7521c8433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.354045 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2a828-e09a-4cdc-8b77-60b7521c8433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.354045 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2a828-e09a-4cdc-8b77-60b7521c8433 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.354516 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2a828-e09a-4cdc-8b77-60b7521c8433 admin admin] PATH_INFO: `/v3/projects/2f63c1ffd86d4bd1aa5989c9bb6d850f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.355880 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2a828-e09a-4cdc-8b77-60b7521c8433 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f63c1ffd86d4bd1aa5989c9bb6d850f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.358067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2a828-e09a-4cdc-8b77-60b7521c8433 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.513842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-88e2a828-e09a-4cdc-8b77-60b7521c8433 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f63c1ffd86d4bd1aa5989c9bb6d850f', 'request_id': 'req-88e2a828-e09a-4cdc-8b77-60b7521c8433'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:24.637240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-88e2a828-e09a-4cdc-8b77-60b7521c8433 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f63c1ffd86d4bd1aa5989c9bb6d850f', 'request_id': 'req-88e2a828-e09a-4cdc-8b77-60b7521c8433'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:24.649055 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-88e2a828-e09a-4cdc-8b77-60b7521c8433 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f63c1ffd86d4bd1aa5989c9bb6d850f', 'request_id': 'req-88e2a828-e09a-4cdc-8b77-60b7521c8433'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:24.657043 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-88e2a828-e09a-4cdc-8b77-60b7521c8433 admin admin] The token cache is being invalidate because project 2f63c1ffd86d4bd1aa5989c9bb6d850f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:24.657329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-88e2a828-e09a-4cdc-8b77-60b7521c8433 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:24.657987 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-88e2a828-e09a-4cdc-8b77-60b7521c8433 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:24.659563 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1116/2199] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:24 2022] DELETE /identity/v3/projects/2f63c1ffd86d4bd1aa5989c9bb6d850f => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:24.664872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d3289-70a2-4349-b96d-739397ba78b3 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.665389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d3289-70a2-4349-b96d-739397ba78b3 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.665489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d3289-70a2-4349-b96d-739397ba78b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.686108 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-554d3289-70a2-4349-b96d-739397ba78b3 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:24.686429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-554d3289-70a2-4349-b96d-739397ba78b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:24.771861 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1084/2200] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:24.780906 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b5ce4-163a-420f-a108-a28da469fa76 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.781197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b5ce4-163a-420f-a108-a28da469fa76 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.781477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b5ce4-163a-420f-a108-a28da469fa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.803434 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c23b5ce4-163a-420f-a108-a28da469fa76 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:24.804964 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c23b5ce4-163a-420f-a108-a28da469fa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:24.813721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1117/2201] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:24.819518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d3289-70a2-4349-b96d-739397ba78b3 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.829392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d3289-70a2-4349-b96d-739397ba78b3 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.836564 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95253ab-6429-4292-b818-ea7063d85364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.837270 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95253ab-6429-4292-b818-ea7063d85364 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.837546 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95253ab-6429-4292-b818-ea7063d85364 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.837817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95253ab-6429-4292-b818-ea7063d85364 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.838465 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95253ab-6429-4292-b818-ea7063d85364 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:24.838759 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95253ab-6429-4292-b818-ea7063d85364 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.840389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95253ab-6429-4292-b818-ea7063d85364 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.852438 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1085/2202] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:02:24.856906 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b5ce4-163a-420f-a108-a28da469fa76 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.868474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b5ce4-163a-420f-a108-a28da469fa76 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.873987 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd5822-7662-403d-b805-8be2691ed311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.874613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd5822-7662-403d-b805-8be2691ed311 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.874920 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd5822-7662-403d-b805-8be2691ed311 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.875231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd5822-7662-403d-b805-8be2691ed311 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.875899 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd5822-7662-403d-b805-8be2691ed311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.878565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd5822-7662-403d-b805-8be2691ed311 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.976567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1118/2203] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:24 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:24.981847 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95253ab-6429-4292-b818-ea7063d85364 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.994718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95253ab-6429-4292-b818-ea7063d85364 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.078214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86974739-9da7-49c6-86be-4f7437ffa453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.079355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86974739-9da7-49c6-86be-4f7437ffa453 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.079495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86974739-9da7-49c6-86be-4f7437ffa453 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.079685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86974739-9da7-49c6-86be-4f7437ffa453 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.082295 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86974739-9da7-49c6-86be-4f7437ffa453 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.082295 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86974739-9da7-49c6-86be-4f7437ffa453 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.153035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1086/2204] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:24 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:25.159095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd5822-7662-403d-b805-8be2691ed311 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.171698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd5822-7662-403d-b805-8be2691ed311 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.178083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e65d26-2c63-45a8-8650-c63611899e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.178825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e65d26-2c63-45a8-8650-c63611899e57 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.179196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e65d26-2c63-45a8-8650-c63611899e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.179590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e65d26-2c63-45a8-8650-c63611899e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.180400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e65d26-2c63-45a8-8650-c63611899e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.182174 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e65d26-2c63-45a8-8650-c63611899e57 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.195991 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1119/2205] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:25.200841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86974739-9da7-49c6-86be-4f7437ffa453 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.214641 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86974739-9da7-49c6-86be-4f7437ffa453 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.220932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefdc36b-724a-423e-a48a-f33ca32c001c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.221558 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdc36b-724a-423e-a48a-f33ca32c001c admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.221839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdc36b-724a-423e-a48a-f33ca32c001c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.222111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdc36b-724a-423e-a48a-f33ca32c001c admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.222956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefdc36b-724a-423e-a48a-f33ca32c001c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.224815 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefdc36b-724a-423e-a48a-f33ca32c001c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.237096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1087/2206] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:25.241467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e65d26-2c63-45a8-8650-c63611899e57 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.252551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e65d26-2c63-45a8-8650-c63611899e57 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.258100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbc7dd-da0f-4f5e-ba13-f3f89a953a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.258300 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbc7dd-da0f-4f5e-ba13-f3f89a953a8e admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.258512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbc7dd-da0f-4f5e-ba13-f3f89a953a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.258789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbc7dd-da0f-4f5e-ba13-f3f89a953a8e admin admin] PATH_INFO: `/v3/projects/c533fdd2bf51465792fc266cefeb58f7/users/29c2921e57dc44d2b6d334e88baae0cd/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.277882 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbc7dd-da0f-4f5e-ba13-f3f89a953a8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c533fdd2bf51465792fc266cefeb58f7, user_id=29c2921e57dc44d2b6d334e88baae0cd, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.280622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbc7dd-da0f-4f5e-ba13-f3f89a953a8e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.284220 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:25.398533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1120/2207] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:25 2022] PUT /identity/v3/projects/c533fdd2bf51465792fc266cefeb58f7/users/29c2921e57dc44d2b6d334e88baae0cd/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:25.403968 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e90ab0-00ab-402c-b49c-864beb0ad85f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.404253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e90ab0-00ab-402c-b49c-864beb0ad85f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.404522 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e90ab0-00ab-402c-b49c-864beb0ad85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.431079 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a6e90ab0-00ab-402c-b49c-864beb0ad85f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:25.432316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a6e90ab0-00ab-402c-b49c-864beb0ad85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:25.468789 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:25.500755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1088/2208] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:25.514097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbc7dd-da0f-4f5e-ba13-f3f89a953a8e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.514097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-cefbc7dd-da0f-4f5e-ba13-f3f89a953a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:25.576084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbc7dd-da0f-4f5e-ba13-f3f89a953a8e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.580509 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27674a3e-835c-48ab-83a8-13e9c8b48c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.581125 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27674a3e-835c-48ab-83a8-13e9c8b48c4d service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.581406 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27674a3e-835c-48ab-83a8-13e9c8b48c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.581673 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27674a3e-835c-48ab-83a8-13e9c8b48c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.591807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27674a3e-835c-48ab-83a8-13e9c8b48c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.593382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27674a3e-835c-48ab-83a8-13e9c8b48c4d service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.620352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1121/2209] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:26.451576 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f508c-e889-4f49-9e99-e76030f241a0 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:26.452524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f508c-e889-4f49-9e99-e76030f241a0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:26.452524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f508c-e889-4f49-9e99-e76030f241a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:26.499281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-457f508c-e889-4f49-9e99-e76030f241a0 None None] MFA Rules not processed for user `beb41eb66e8140a6999b0bddab3c4afd`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:26.500563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-457f508c-e889-4f49-9e99-e76030f241a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:26.589619 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1089/2210] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:02:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:32.677743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c501d-608f-4773-b70b-c3687954be6d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:32.678542 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c501d-608f-4773-b70b-c3687954be6d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:32.678827 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c501d-608f-4773-b70b-c3687954be6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:32.728827 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ae4c501d-608f-4773-b70b-c3687954be6d None None] MFA Rules not processed for user `29c2921e57dc44d2b6d334e88baae0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:32.730187 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ae4c501d-608f-4773-b70b-c3687954be6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:32.841698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1122/2211] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:02:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:32.846739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f508c-e889-4f49-9e99-e76030f241a0 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:32.858011 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f508c-e889-4f49-9e99-e76030f241a0 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:32.863344 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400fa78-ffe5-4b16-84f7-4d9159e28c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:32.863958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400fa78-ffe5-4b16-84f7-4d9159e28c99 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:32.864514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400fa78-ffe5-4b16-84f7-4d9159e28c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:32.864876 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400fa78-ffe5-4b16-84f7-4d9159e28c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:32.865773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400fa78-ffe5-4b16-84f7-4d9159e28c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:32.867879 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6400fa78-ffe5-4b16-84f7-4d9159e28c99 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.017814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1090/2212] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:32 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:33.021852 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c501d-608f-4773-b70b-c3687954be6d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:33.034642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c501d-608f-4773-b70b-c3687954be6d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.108026 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f03f4-6302-4f98-b8bd-b6040d787682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.108652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f03f4-6302-4f98-b8bd-b6040d787682 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.108934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f03f4-6302-4f98-b8bd-b6040d787682 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.109209 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f03f4-6302-4f98-b8bd-b6040d787682 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.109932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f03f4-6302-4f98-b8bd-b6040d787682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.111946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f03f4-6302-4f98-b8bd-b6040d787682 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.290722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1123/2213] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:33 2022] POST /identity/v3/users => generated 298 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:33.296642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400fa78-ffe5-4b16-84f7-4d9159e28c99 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:33.307879 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6400fa78-ffe5-4b16-84f7-4d9159e28c99 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.312930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0249c93-0912-4163-9bdb-0d40f73bf2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.313690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0249c93-0912-4163-9bdb-0d40f73bf2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.314733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0249c93-0912-4163-9bdb-0d40f73bf2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.314733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0249c93-0912-4163-9bdb-0d40f73bf2b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.315572 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0249c93-0912-4163-9bdb-0d40f73bf2b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.317019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0249c93-0912-4163-9bdb-0d40f73bf2b1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.330164 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1091/2214] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:33.333899 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f03f4-6302-4f98-b8bd-b6040d787682 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:33.347986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f03f4-6302-4f98-b8bd-b6040d787682 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.353303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39032c13-8679-4155-9f50-815512ee58bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.353998 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39032c13-8679-4155-9f50-815512ee58bc admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.354307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39032c13-8679-4155-9f50-815512ee58bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.354621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39032c13-8679-4155-9f50-815512ee58bc admin admin] PATH_INFO: `/v3/projects/54a65d7e2018462da3c8506472250952/users/f577ab5339c148c29416ebcd4483aa43/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.377907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:33.380214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39032c13-8679-4155-9f50-815512ee58bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a65d7e2018462da3c8506472250952, user_id=f577ab5339c148c29416ebcd4483aa43, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.383620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39032c13-8679-4155-9f50-815512ee58bc admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.525007 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1124/2215] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:33 2022] PUT /identity/v3/projects/54a65d7e2018462da3c8506472250952/users/f577ab5339c148c29416ebcd4483aa43/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:33.529090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0249c93-0912-4163-9bdb-0d40f73bf2b1 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:33.539422 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0249c93-0912-4163-9bdb-0d40f73bf2b1 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.598938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c1e199-e52c-4aaa-a6ed-150fca5a6689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.599510 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1e199-e52c-4aaa-a6ed-150fca5a6689 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.599804 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1e199-e52c-4aaa-a6ed-150fca5a6689 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.600058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c1e199-e52c-4aaa-a6ed-150fca5a6689 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.600693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c1e199-e52c-4aaa-a6ed-150fca5a6689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.605247 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c1e199-e52c-4aaa-a6ed-150fca5a6689 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.622079 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1092/2216] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:33.623721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39032c13-8679-4155-9f50-815512ee58bc admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:33.636247 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39032c13-8679-4155-9f50-815512ee58bc admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.659790 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6443f4-88b6-4e58-8986-cfdac879c872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.661134 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6443f4-88b6-4e58-8986-cfdac879c872 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.661547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6443f4-88b6-4e58-8986-cfdac879c872 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.661894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6443f4-88b6-4e58-8986-cfdac879c872 admin admin] PATH_INFO: `/v3/projects/54a65d7e2018462da3c8506472250952/users/f577ab5339c148c29416ebcd4483aa43/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.665842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6443f4-88b6-4e58-8986-cfdac879c872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a65d7e2018462da3c8506472250952, user_id=f577ab5339c148c29416ebcd4483aa43, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.669760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6443f4-88b6-4e58-8986-cfdac879c872 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.690676 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1125/2217] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:33 2022] PUT /identity/v3/projects/54a65d7e2018462da3c8506472250952/users/f577ab5339c148c29416ebcd4483aa43/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:37.056552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffba37f-050d-45f2-8bb0-7cb012c71dd0 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:37.057121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffba37f-050d-45f2-8bb0-7cb012c71dd0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:37.057312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffba37f-050d-45f2-8bb0-7cb012c71dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:37.096400 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1ffba37f-050d-45f2-8bb0-7cb012c71dd0 None None] MFA Rules not processed for user `e4916206c2ff42669276e0a93d6483ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:37.101674 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1ffba37f-050d-45f2-8bb0-7cb012c71dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:37.166609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:37.186079 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1093/2218] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:02:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:40.915494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26dc45-77de-43f0-87b8-488ac083460d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:40.915979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26dc45-77de-43f0-87b8-488ac083460d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:40.916055 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26dc45-77de-43f0-87b8-488ac083460d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.003926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1d26dc45-77de-43f0-87b8-488ac083460d None None] MFA Rules not processed for user `f577ab5339c148c29416ebcd4483aa43`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:41.007877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1d26dc45-77de-43f0-87b8-488ac083460d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:41.195827 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1126/2219] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:02:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:41.211881 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffba37f-050d-45f2-8bb0-7cb012c71dd0 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.215222 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1ffba37f-050d-45f2-8bb0-7cb012c71dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:41.286175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffba37f-050d-45f2-8bb0-7cb012c71dd0 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.290386 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05344261-de83-472a-a9f3-c4ad6ead166e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b060c70e1b440186eeac37d9c34567', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.291059 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05344261-de83-472a-a9f3-c4ad6ead166e service glance] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.291336 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05344261-de83-472a-a9f3-c4ad6ead166e service glance] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.291610 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05344261-de83-472a-a9f3-c4ad6ead166e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.397499 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05344261-de83-472a-a9f3-c4ad6ead166e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.398994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05344261-de83-472a-a9f3-c4ad6ead166e service glance] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:41.420685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1094/2220] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:41.798709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33796c67-e5e8-46a8-8993-5a050f0cbec7 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.799060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33796c67-e5e8-46a8-8993-5a050f0cbec7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.799337 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33796c67-e5e8-46a8-8993-5a050f0cbec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.819526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-33796c67-e5e8-46a8-8993-5a050f0cbec7 None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:41.820804 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-33796c67-e5e8-46a8-8993-5a050f0cbec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:41.877065 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1127/2221] 10.222.0.91 () {58 vars in 1009 bytes} [Thu Sep 8 14:02:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:41.889356 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05344261-de83-472a-a9f3-c4ad6ead166e service glance] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.890763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-05344261-de83-472a-a9f3-c4ad6ead166e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:41.964854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05344261-de83-472a-a9f3-c4ad6ead166e service glance] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.971175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62abaf-661b-4e81-b0a0-db00bed2104a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.971729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62abaf-661b-4e81-b0a0-db00bed2104a service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.972009 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62abaf-661b-4e81-b0a0-db00bed2104a service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.972278 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62abaf-661b-4e81-b0a0-db00bed2104a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.982261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62abaf-661b-4e81-b0a0-db00bed2104a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.984132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62abaf-661b-4e81-b0a0-db00bed2104a service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:41.993368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1095/2222] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 14:02:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:02:46.286472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33796c67-e5e8-46a8-8993-5a050f0cbec7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:46.289406 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-33796c67-e5e8-46a8-8993-5a050f0cbec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:46.494847 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33796c67-e5e8-46a8-8993-5a050f0cbec7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:46.503724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0a7e1-f0a7-45e5-8d5d-018e9d5d0fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:46.504568 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0a7e1-f0a7-45e5-8d5d-018e9d5d0fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:46.504950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0a7e1-f0a7-45e5-8d5d-018e9d5d0fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:46.505865 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0a7e1-f0a7-45e5-8d5d-018e9d5d0fe0 admin admin] PATH_INFO: `/v3/users/345303d8b254401189917dac0eaa2905` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:46.507250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0a7e1-f0a7-45e5-8d5d-018e9d5d0fe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=345303d8b254401189917dac0eaa2905)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:46.509999 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0a7e1-f0a7-45e5-8d5d-018e9d5d0fe0 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:46.536084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:46.836745 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c1a0a7e1-f0a7-45e5-8d5d-018e9d5d0fe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '345303d8b254401189917dac0eaa2905'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:46.969042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c1a0a7e1-f0a7-45e5-8d5d-018e9d5d0fe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '345303d8b254401189917dac0eaa2905'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:46.969861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c1a0a7e1-f0a7-45e5-8d5d-018e9d5d0fe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '345303d8b254401189917dac0eaa2905'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:46.986825 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c1a0a7e1-f0a7-45e5-8d5d-018e9d5d0fe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '345303d8b254401189917dac0eaa2905'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:47.045532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c1a0a7e1-f0a7-45e5-8d5d-018e9d5d0fe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '345303d8b254401189917dac0eaa2905'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:47.066969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1128/2223] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:46 2022] DELETE /identity/v3/users/345303d8b254401189917dac0eaa2905 => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:47.069197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c69c0-17af-4a12-b7e9-d010596f1853 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:47.069977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c69c0-17af-4a12-b7e9-d010596f1853 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:47.070087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c69c0-17af-4a12-b7e9-d010596f1853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:47.154430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-739c69c0-17af-4a12-b7e9-d010596f1853 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:47.156125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-739c69c0-17af-4a12-b7e9-d010596f1853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:47.315658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:47.427243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1096/2224] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:47.445595 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0a7e1-f0a7-45e5-8d5d-018e9d5d0fe0 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:47.447118 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c1a0a7e1-f0a7-45e5-8d5d-018e9d5d0fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:47.570712 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0a7e1-f0a7-45e5-8d5d-018e9d5d0fe0 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:47.577113 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640e239-912d-44b1-98ab-fb5e6419eb6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:47.578292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640e239-912d-44b1-98ab-fb5e6419eb6d service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:47.579513 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640e239-912d-44b1-98ab-fb5e6419eb6d service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:47.580029 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640e239-912d-44b1-98ab-fb5e6419eb6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:47.598057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640e239-912d-44b1-98ab-fb5e6419eb6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:47.600257 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640e239-912d-44b1-98ab-fb5e6419eb6d service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:47.614447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1129/2225] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:48.915885 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c69c0-17af-4a12-b7e9-d010596f1853 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:48.918763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-739c69c0-17af-4a12-b7e9-d010596f1853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:48.959603 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c69c0-17af-4a12-b7e9-d010596f1853 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:48.976719 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192120d4-7e98-4f9e-b599-c39e92d21718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:48.984060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192120d4-7e98-4f9e-b599-c39e92d21718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:48.985108 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192120d4-7e98-4f9e-b599-c39e92d21718 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:48.988234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192120d4-7e98-4f9e-b599-c39e92d21718 admin admin] PATH_INFO: `/v3/projects/4666b3c98e7b4830a798fb30631140eb` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:48.991752 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192120d4-7e98-4f9e-b599-c39e92d21718 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4666b3c98e7b4830a798fb30631140eb)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:48.999937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192120d4-7e98-4f9e-b599-c39e92d21718 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:49.287874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-192120d4-7e98-4f9e-b599-c39e92d21718 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4666b3c98e7b4830a798fb30631140eb', 'request_id': 'req-192120d4-7e98-4f9e-b599-c39e92d21718'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:49.296111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-192120d4-7e98-4f9e-b599-c39e92d21718 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4666b3c98e7b4830a798fb30631140eb', 'request_id': 'req-192120d4-7e98-4f9e-b599-c39e92d21718'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:49.309175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-192120d4-7e98-4f9e-b599-c39e92d21718 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4666b3c98e7b4830a798fb30631140eb', 'request_id': 'req-192120d4-7e98-4f9e-b599-c39e92d21718'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:49.527806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-192120d4-7e98-4f9e-b599-c39e92d21718 admin admin] The token cache is being invalidate because project 4666b3c98e7b4830a798fb30631140eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:49.527806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-192120d4-7e98-4f9e-b599-c39e92d21718 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:49.527806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-192120d4-7e98-4f9e-b599-c39e92d21718 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:49.527806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1097/2226] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:48 2022] DELETE /identity/v3/projects/4666b3c98e7b4830a798fb30631140eb => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:57.472603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5e6bc-6216-48a9-9ce5-b4247b16c9b5 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:57.482192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5e6bc-6216-48a9-9ce5-b4247b16c9b5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:57.482192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5e6bc-6216-48a9-9ce5-b4247b16c9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:57.496504 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-eaa5e6bc-6216-48a9-9ce5-b4247b16c9b5 None None] MFA Rules not processed for user `29c2921e57dc44d2b6d334e88baae0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:57.497760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-eaa5e6bc-6216-48a9-9ce5-b4247b16c9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:57.567802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1130/2227] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:02:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:59.517679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:00.656963 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192120d4-7e98-4f9e-b599-c39e92d21718 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:00.658535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-192120d4-7e98-4f9e-b599-c39e92d21718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:00.729721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192120d4-7e98-4f9e-b599-c39e92d21718 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:00.734791 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe58ede-2e79-496b-bcca-13ead5d8e533 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9506139d5744fe69153e7e30a05b436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:00.735424 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe58ede-2e79-496b-bcca-13ead5d8e533 service placement] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:00.735702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe58ede-2e79-496b-bcca-13ead5d8e533 service placement] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:00.736463 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe58ede-2e79-496b-bcca-13ead5d8e533 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:00.738181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2fe58ede-2e79-496b-bcca-13ead5d8e533 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:00.749779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe58ede-2e79-496b-bcca-13ead5d8e533 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:00.752025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe58ede-2e79-496b-bcca-13ead5d8e533 service placement] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:00.762193 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1098/2228] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:03:03.078314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fc5e0-cad6-49b8-bd6a-9bdd60348be5 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:03.080841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fc5e0-cad6-49b8-bd6a-9bdd60348be5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:03.080841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fc5e0-cad6-49b8-bd6a-9bdd60348be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:03.100212 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-7a8fc5e0-cad6-49b8-bd6a-9bdd60348be5 None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:03.101754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7a8fc5e0-cad6-49b8-bd6a-9bdd60348be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:03.162593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:03.201680 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1131/2229] 10.222.0.91 () {58 vars in 1009 bytes} [Thu Sep 8 14:03:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:03.218195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe58ede-2e79-496b-bcca-13ead5d8e533 service placement] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:03.219790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2fe58ede-2e79-496b-bcca-13ead5d8e533 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:03.313754 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe58ede-2e79-496b-bcca-13ead5d8e533 service placement] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:03.321810 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d32e88-790e-42e3-a734-ee834ee8f3bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:03.322853 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d32e88-790e-42e3-a734-ee834ee8f3bc service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:03.323768 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d32e88-790e-42e3-a734-ee834ee8f3bc service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:03.323768 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d32e88-790e-42e3-a734-ee834ee8f3bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:03.336798 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d32e88-790e-42e3-a734-ee834ee8f3bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:03.339936 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d32e88-790e-42e3-a734-ee834ee8f3bc service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:03.350718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1099/2230] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 14:03:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:03:13.347728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=64 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:13.693257 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=9 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:31.025664 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660ab61-75aa-4a96-89ac-973d5242b10d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:31.026293 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660ab61-75aa-4a96-89ac-973d5242b10d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:31.026293 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660ab61-75aa-4a96-89ac-973d5242b10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:31.045982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-7660ab61-75aa-4a96-89ac-973d5242b10d None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:31.047301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7660ab61-75aa-4a96-89ac-973d5242b10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:31.055800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1132/2231] 10.222.0.91 () {58 vars in 1009 bytes} [Thu Sep 8 14:03:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:31.080680 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d32e88-790e-42e3-a734-ee834ee8f3bc service swift] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:31.091990 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d32e88-790e-42e3-a734-ee834ee8f3bc service swift] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:31.098181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e16d9-c614-4ca3-b32f-53f78b2a8be4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:31.098886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e16d9-c614-4ca3-b32f-53f78b2a8be4 service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:31.099182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e16d9-c614-4ca3-b32f-53f78b2a8be4 service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:31.099452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e16d9-c614-4ca3-b32f-53f78b2a8be4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:31.107840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e16d9-c614-4ca3-b32f-53f78b2a8be4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:31.109485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e16d9-c614-4ca3-b32f-53f78b2a8be4 service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:31.118761 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1100/2232] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 14:03:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:03:38.700589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660ab61-75aa-4a96-89ac-973d5242b10d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:38.702491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7660ab61-75aa-4a96-89ac-973d5242b10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:38.774997 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660ab61-75aa-4a96-89ac-973d5242b10d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:38.779114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fbd82-6390-496f-97fb-dc9b04ea23f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:38.779975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fbd82-6390-496f-97fb-dc9b04ea23f8 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:38.780455 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fbd82-6390-496f-97fb-dc9b04ea23f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:38.780927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fbd82-6390-496f-97fb-dc9b04ea23f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:38.784235 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-211fbd82-6390-496f-97fb-dc9b04ea23f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:38.870588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211fbd82-6390-496f-97fb-dc9b04ea23f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:38.872294 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211fbd82-6390-496f-97fb-dc9b04ea23f8 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:38.884047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1133/2233] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:41.115281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=6 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:48.879199 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=53 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:51.995121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de128ab-b592-40b1-a448-624406095cd0 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:51.995599 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de128ab-b592-40b1-a448-624406095cd0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:51.995705 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de128ab-b592-40b1-a448-624406095cd0 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:51.997165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1101/2234] 10.222.0.91 () {54 vars in 921 bytes} [Thu Sep 8 14:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:03:52.010541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820ebf5-c32b-42a1-8db4-ce2d48d6ec39 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:52.011236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820ebf5-c32b-42a1-8db4-ce2d48d6ec39 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:52.011236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820ebf5-c32b-42a1-8db4-ce2d48d6ec39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:52.068220 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-0820ebf5-c32b-42a1-8db4-ce2d48d6ec39 None None] MFA Rules not processed for user `38127f41852d412ea6ad270f1618fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:52.070826 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0820ebf5-c32b-42a1-8db4-ce2d48d6ec39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:52.185848 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1134/2235] 10.222.0.91 () {58 vars in 1018 bytes} [Thu Sep 8 14:03:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:52.199684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de128ab-b592-40b1-a448-624406095cd0 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:52.202592 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4de128ab-b592-40b1-a448-624406095cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:52.276342 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de128ab-b592-40b1-a448-624406095cd0 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:52.280577 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2d56b-17b3-4a78-9612-f03dbb83bc52 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b060c70e1b440186eeac37d9c34567', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:52.281822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2d56b-17b3-4a78-9612-f03dbb83bc52 service glance] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:52.282283 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2d56b-17b3-4a78-9612-f03dbb83bc52 service glance] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:52.282719 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2d56b-17b3-4a78-9612-f03dbb83bc52 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:52.300251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2d56b-17b3-4a78-9612-f03dbb83bc52 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:52.302272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2d56b-17b3-4a78-9612-f03dbb83bc52 service glance] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:52.316779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1102/2236] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:52.548313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacf099-1d7c-4524-8c67-f6b5547c04c1 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:52.548602 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacf099-1d7c-4524-8c67-f6b5547c04c1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:52.548874 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacf099-1d7c-4524-8c67-f6b5547c04c1 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:52.550276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1135/2237] 10.222.0.91 () {54 vars in 921 bytes} [Thu Sep 8 14:03:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:03:52.559525 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d073a-68c6-4401-a0ac-05d5fa4eff21 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:52.559812 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d073a-68c6-4401-a0ac-05d5fa4eff21 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:52.560089 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d073a-68c6-4401-a0ac-05d5fa4eff21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:52.581824 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9f8d073a-68c6-4401-a0ac-05d5fa4eff21 None None] MFA Rules not processed for user `cdbbaa64ad4844a0b2698dee82f95f23`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:52.583140 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9f8d073a-68c6-4401-a0ac-05d5fa4eff21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:52.589919 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1103/2238] 10.222.0.91 () {58 vars in 1018 bytes} [Thu Sep 8 14:03:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:52.602712 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacf099-1d7c-4524-8c67-f6b5547c04c1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:52.615661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacf099-1d7c-4524-8c67-f6b5547c04c1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:52.620871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16db87-59e1-42c4-90cf-6c1a86d1d60b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:52.621503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16db87-59e1-42c4-90cf-6c1a86d1d60b service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:52.626538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16db87-59e1-42c4-90cf-6c1a86d1d60b service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:52.626538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16db87-59e1-42c4-90cf-6c1a86d1d60b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:52.631500 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16db87-59e1-42c4-90cf-6c1a86d1d60b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:52.633034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16db87-59e1-42c4-90cf-6c1a86d1d60b service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:52.642790 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1136/2239] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:52.740117 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4ff83-5b16-4701-bf61-9b73625e100f None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:52.740577 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4ff83-5b16-4701-bf61-9b73625e100f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:52.741045 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4ff83-5b16-4701-bf61-9b73625e100f None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:52.747747 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1104/2240] 10.222.0.91 () {54 vars in 921 bytes} [Thu Sep 8 14:03:52 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:03:52.755651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8735e4-b061-4499-8db4-260013763838 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:52.755887 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8735e4-b061-4499-8db4-260013763838 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:52.756190 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8735e4-b061-4499-8db4-260013763838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:52.777035 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-df8735e4-b061-4499-8db4-260013763838 None None] MFA Rules not processed for user `38127f41852d412ea6ad270f1618fce2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:52.778253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-df8735e4-b061-4499-8db4-260013763838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:52.787781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1137/2241] 10.222.0.91 () {58 vars in 1018 bytes} [Thu Sep 8 14:03:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:52.800361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4ff83-5b16-4701-bf61-9b73625e100f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:52.802173 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ade4ff83-5b16-4701-bf61-9b73625e100f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:52.818927 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4ff83-5b16-4701-bf61-9b73625e100f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:52.822813 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddebf4-434c-4ea9-a82e-4f6cba80ce37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:52.823450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddebf4-434c-4ea9-a82e-4f6cba80ce37 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:52.823729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddebf4-434c-4ea9-a82e-4f6cba80ce37 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:52.824022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddebf4-434c-4ea9-a82e-4f6cba80ce37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:52.836930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bddebf4-434c-4ea9-a82e-4f6cba80ce37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:52.838390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bddebf4-434c-4ea9-a82e-4f6cba80ce37 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:52.850661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1105/2242] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:59.552569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e00ef6-a53d-4b42-b8d2-1a97b451a4d1 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:59.553081 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e00ef6-a53d-4b42-b8d2-1a97b451a4d1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:59.553189 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e00ef6-a53d-4b42-b8d2-1a97b451a4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:59.579027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-63e00ef6-a53d-4b42-b8d2-1a97b451a4d1 None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:59.579907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-63e00ef6-a53d-4b42-b8d2-1a97b451a4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:59.587408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1138/2243] 10.222.0.91 () {58 vars in 1009 bytes} [Thu Sep 8 14:03:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:59.603387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddebf4-434c-4ea9-a82e-4f6cba80ce37 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:59.616550 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddebf4-434c-4ea9-a82e-4f6cba80ce37 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:59.624504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceeb333-3bbd-49b3-bced-5a0943698606 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:59.625576 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceeb333-3bbd-49b3-bced-5a0943698606 service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:59.626069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceeb333-3bbd-49b3-bced-5a0943698606 service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:59.626551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceeb333-3bbd-49b3-bced-5a0943698606 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:59.640163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceeb333-3bbd-49b3-bced-5a0943698606 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:59.641828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceeb333-3bbd-49b3-bced-5a0943698606 service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:59.650860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1106/2244] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 14:03:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:04:09.572175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=43 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:09.652039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=41 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:15.243074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7a883-9fef-49d4-b14c-2423ee7cb274 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:15.243936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7a883-9fef-49d4-b14c-2423ee7cb274 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:15.243936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7a883-9fef-49d4-b14c-2423ee7cb274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:15.265308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-70b7a883-9fef-49d4-b14c-2423ee7cb274 None None] MFA Rules not processed for user `beb41eb66e8140a6999b0bddab3c4afd`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:15.271200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-70b7a883-9fef-49d4-b14c-2423ee7cb274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:15.328598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1139/2245] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:04:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:04:22.683393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86801b01-964c-4386-96e2-81051a590cb5 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:22.683912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86801b01-964c-4386-96e2-81051a590cb5 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:22.684022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86801b01-964c-4386-96e2-81051a590cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:22.705284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-86801b01-964c-4386-96e2-81051a590cb5 None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:22.706784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-86801b01-964c-4386-96e2-81051a590cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:22.713898 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1107/2246] 10.222.0.91 () {58 vars in 967 bytes} [Thu Sep 8 14:04:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:22.726465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7a883-9fef-49d4-b14c-2423ee7cb274 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:22.739239 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7a883-9fef-49d4-b14c-2423ee7cb274 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:22.744511 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4ccb0-56ec-4c48-926c-2746640dccf2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:22.745129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4ccb0-56ec-4c48-926c-2746640dccf2 service swift] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:22.745409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4ccb0-56ec-4c48-926c-2746640dccf2 service swift] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:22.745683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4ccb0-56ec-4c48-926c-2746640dccf2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:22.756527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4ccb0-56ec-4c48-926c-2746640dccf2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:22.757877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e4ccb0-56ec-4c48-926c-2746640dccf2 service swift] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:22.767386 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1140/2247] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 14:04:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:04:32.699332 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:32.764119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=31 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:40.063280 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86801b01-964c-4386-96e2-81051a590cb5 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:40.073003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86801b01-964c-4386-96e2-81051a590cb5 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:40.076729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a510230-8edc-4977-8082-225b85da5a23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:40.077279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a510230-8edc-4977-8082-225b85da5a23 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:40.078107 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a510230-8edc-4977-8082-225b85da5a23 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:40.078385 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a510230-8edc-4977-8082-225b85da5a23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:40.080190 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7a510230-8edc-4977-8082-225b85da5a23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:40.094096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a510230-8edc-4977-8082-225b85da5a23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:40.095559 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a510230-8edc-4977-8082-225b85da5a23 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:40.106607 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1108/2248] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:41.294185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4ccb0-56ec-4c48-926c-2746640dccf2 service swift] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:41.303678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4ccb0-56ec-4c48-926c-2746640dccf2 service swift] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:41.307309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795cf5e8-6f36-433f-95fa-0be57b4f922f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:41.307908 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795cf5e8-6f36-433f-95fa-0be57b4f922f service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:41.308186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795cf5e8-6f36-433f-95fa-0be57b4f922f service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:41.308471 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795cf5e8-6f36-433f-95fa-0be57b4f922f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:41.310170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-795cf5e8-6f36-433f-95fa-0be57b4f922f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:41.328440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795cf5e8-6f36-433f-95fa-0be57b4f922f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:41.329957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795cf5e8-6f36-433f-95fa-0be57b4f922f service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:41.341617 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1141/2249] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:47.459519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441179d-f59e-4db1-b3c0-28dae3122cb4 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:47.460098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441179d-f59e-4db1-b3c0-28dae3122cb4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:47.460098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441179d-f59e-4db1-b3c0-28dae3122cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:47.481377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c441179d-f59e-4db1-b3c0-28dae3122cb4 None None] MFA Rules not processed for user `e4916206c2ff42669276e0a93d6483ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:47.482609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c441179d-f59e-4db1-b3c0-28dae3122cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:47.541243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1109/2250] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:04:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:04:51.337366 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=6 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:53.020139 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795cf5e8-6f36-433f-95fa-0be57b4f922f service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:53.021674 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-795cf5e8-6f36-433f-95fa-0be57b4f922f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:53.089829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795cf5e8-6f36-433f-95fa-0be57b4f922f service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:53.094147 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe41400-759b-4675-86a8-851e6444ef45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:53.095113 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe41400-759b-4675-86a8-851e6444ef45 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:53.095563 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe41400-759b-4675-86a8-851e6444ef45 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:53.096013 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe41400-759b-4675-86a8-851e6444ef45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:53.098093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-dfe41400-759b-4675-86a8-851e6444ef45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:53.171454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe41400-759b-4675-86a8-851e6444ef45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:53.176221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe41400-759b-4675-86a8-851e6444ef45 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:53.185890 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1142/2251] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:57.534355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=31 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:03.179691 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=48 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:03.438603 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f464b85-069e-418c-bdea-2d3ae9b1d908 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:03.439230 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f464b85-069e-418c-bdea-2d3ae9b1d908 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:03.439230 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f464b85-069e-418c-bdea-2d3ae9b1d908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:03.458447 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-5f464b85-069e-418c-bdea-2d3ae9b1d908 None None] MFA Rules not processed for user `7b403bdc9cad4783b72bbf174199363c`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:03.459678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5f464b85-069e-418c-bdea-2d3ae9b1d908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:03.466786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1110/2252] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:05:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:05:03.782939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe41400-759b-4675-86a8-851e6444ef45 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:03.792875 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe41400-759b-4675-86a8-851e6444ef45 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:03.796769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1a43e-7e4f-450e-ad38-69e2b77dd6e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:03.797376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1a43e-7e4f-450e-ad38-69e2b77dd6e7 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:03.797657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1a43e-7e4f-450e-ad38-69e2b77dd6e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:03.797930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1a43e-7e4f-450e-ad38-69e2b77dd6e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:03.799850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-48e1a43e-7e4f-450e-ad38-69e2b77dd6e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:03.810655 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1a43e-7e4f-450e-ad38-69e2b77dd6e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:03.812066 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1a43e-7e4f-450e-ad38-69e2b77dd6e7 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:03.820887 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1143/2253] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:04.326198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f464b85-069e-418c-bdea-2d3ae9b1d908 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:04.338846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f464b85-069e-418c-bdea-2d3ae9b1d908 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:04.343834 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35dacd-43e1-4aa3-b1a4-3c47dc2973e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9506139d5744fe69153e7e30a05b436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:04.344436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35dacd-43e1-4aa3-b1a4-3c47dc2973e2 service placement] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:04.344711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35dacd-43e1-4aa3-b1a4-3c47dc2973e2 service placement] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:04.344978 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35dacd-43e1-4aa3-b1a4-3c47dc2973e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:04.346701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0d35dacd-43e1-4aa3-b1a4-3c47dc2973e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:04.359334 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35dacd-43e1-4aa3-b1a4-3c47dc2973e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:04.360779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35dacd-43e1-4aa3-b1a4-3c47dc2973e2 service placement] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:04.371927 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1111/2254] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:13.817237 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=6 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:14.366795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=9 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:16.917131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1a43e-7e4f-450e-ad38-69e2b77dd6e7 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:16.926467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1a43e-7e4f-450e-ad38-69e2b77dd6e7 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:16.930545 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184ddda-3742-49e1-a402-3fd6fb111ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:16.931493 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ddda-3742-49e1-a402-3fd6fb111ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:16.931778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ddda-3742-49e1-a402-3fd6fb111ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:16.932066 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ddda-3742-49e1-a402-3fd6fb111ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:16.933846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3184ddda-3742-49e1-a402-3fd6fb111ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:16.944852 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184ddda-3742-49e1-a402-3fd6fb111ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:16.946325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184ddda-3742-49e1-a402-3fd6fb111ba9 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:16.958304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1144/2255] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:19.576186 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35dacd-43e1-4aa3-b1a4-3c47dc2973e2 service placement] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:19.577854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0d35dacd-43e1-4aa3-b1a4-3c47dc2973e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:19.592749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35dacd-43e1-4aa3-b1a4-3c47dc2973e2 service placement] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:19.599266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae7a29-4fdc-4186-aea0-6e9befd02e5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:19.599873 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae7a29-4fdc-4186-aea0-6e9befd02e5f service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:19.600152 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae7a29-4fdc-4186-aea0-6e9befd02e5f service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:19.600424 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae7a29-4fdc-4186-aea0-6e9befd02e5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:19.602172 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-efae7a29-4fdc-4186-aea0-6e9befd02e5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:19.613892 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae7a29-4fdc-4186-aea0-6e9befd02e5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:19.615520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae7a29-4fdc-4186-aea0-6e9befd02e5f service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:19.628005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1112/2256] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:22.363858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5ab3-83cc-4d6b-9f41-64ca2e99b5ac None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:22.364415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5ab3-83cc-4d6b-9f41-64ca2e99b5ac None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:22.364597 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5ab3-83cc-4d6b-9f41-64ca2e99b5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:22.384598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-80ca5ab3-83cc-4d6b-9f41-64ca2e99b5ac None None] MFA Rules not processed for user `48bf6dab289545bb80182c8904c05d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:22.385803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-80ca5ab3-83cc-4d6b-9f41-64ca2e99b5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:22.444264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1145/2257] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:05:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:05:22.779972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae7a29-4fdc-4186-aea0-6e9befd02e5f service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:22.789699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae7a29-4fdc-4186-aea0-6e9befd02e5f service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:22.793749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350fe0b-b787-40ae-bb4b-aa79bf8076a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:22.794303 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350fe0b-b787-40ae-bb4b-aa79bf8076a9 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:22.794583 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350fe0b-b787-40ae-bb4b-aa79bf8076a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:22.794853 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350fe0b-b787-40ae-bb4b-aa79bf8076a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:22.796641 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7350fe0b-b787-40ae-bb4b-aa79bf8076a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:22.807180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350fe0b-b787-40ae-bb4b-aa79bf8076a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:22.808611 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350fe0b-b787-40ae-bb4b-aa79bf8076a9 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:22.817150 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1113/2258] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:23.793198 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca5ab3-83cc-4d6b-9f41-64ca2e99b5ac None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:23.794866 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-80ca5ab3-83cc-4d6b-9f41-64ca2e99b5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:23.821205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca5ab3-83cc-4d6b-9f41-64ca2e99b5ac None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:23.827719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d76ff-c03c-4f35-8c0d-20597fec2fdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:23.828858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d76ff-c03c-4f35-8c0d-20597fec2fdb service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:23.829231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d76ff-c03c-4f35-8c0d-20597fec2fdb service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:23.829597 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d76ff-c03c-4f35-8c0d-20597fec2fdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:23.831519 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-529d76ff-c03c-4f35-8c0d-20597fec2fdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:23.845246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d76ff-c03c-4f35-8c0d-20597fec2fdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:23.847084 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d76ff-c03c-4f35-8c0d-20597fec2fdb service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:23.872520 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1146/2259] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:24.714504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350fe0b-b787-40ae-bb4b-aa79bf8076a9 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:24.739001 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350fe0b-b787-40ae-bb4b-aa79bf8076a9 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:24.744775 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b81d39-fbf0-4b12-9fcd-af68f678af02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:24.745562 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b81d39-fbf0-4b12-9fcd-af68f678af02 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:24.745665 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b81d39-fbf0-4b12-9fcd-af68f678af02 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:24.745901 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b81d39-fbf0-4b12-9fcd-af68f678af02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:24.747699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e7b81d39-fbf0-4b12-9fcd-af68f678af02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:24.763684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b81d39-fbf0-4b12-9fcd-af68f678af02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:24.765394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b81d39-fbf0-4b12-9fcd-af68f678af02 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:24.777880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1114/2260] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:26.932551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a715263-1145-470d-b0ed-5c6f40cb9df5 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:26.936894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a715263-1145-470d-b0ed-5c6f40cb9df5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:26.936894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a715263-1145-470d-b0ed-5c6f40cb9df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:26.956737 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-9a715263-1145-470d-b0ed-5c6f40cb9df5 None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:26.958443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9a715263-1145-470d-b0ed-5c6f40cb9df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:26.966073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1147/2261] 10.222.0.91 () {58 vars in 967 bytes} [Thu Sep 8 14:05:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:05:27.184058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b81d39-fbf0-4b12-9fcd-af68f678af02 service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:27.196762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b81d39-fbf0-4b12-9fcd-af68f678af02 service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:27.201977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018391a-d8fb-485e-b1b8-c9c93bfcf214 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:27.202687 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018391a-d8fb-485e-b1b8-c9c93bfcf214 service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:27.202856 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018391a-d8fb-485e-b1b8-c9c93bfcf214 service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:27.203156 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018391a-d8fb-485e-b1b8-c9c93bfcf214 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:27.212704 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018391a-d8fb-485e-b1b8-c9c93bfcf214 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:27.214279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018391a-d8fb-485e-b1b8-c9c93bfcf214 service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:27.222640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1115/2262] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 14:05:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:05:33.234736 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d244a7-0395-4f82-956f-4da74495fc41 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:33.240701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d244a7-0395-4f82-956f-4da74495fc41 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:33.241124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d244a7-0395-4f82-956f-4da74495fc41 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:33.244823 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1148/2263] 10.222.0.91 () {54 vars in 961 bytes} [Thu Sep 8 14:05:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:05:33.250479 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094750d-a938-4cde-b3f5-10dd8f7e9dda None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:33.251074 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094750d-a938-4cde-b3f5-10dd8f7e9dda None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:33.251175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094750d-a938-4cde-b3f5-10dd8f7e9dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:33.279520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4094750d-a938-4cde-b3f5-10dd8f7e9dda None None] MFA Rules not processed for user `f9506139d5744fe69153e7e30a05b436`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:33.281073 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4094750d-a938-4cde-b3f5-10dd8f7e9dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:33.295783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1116/2264] 10.222.0.91 () {58 vars in 1058 bytes} [Thu Sep 8 14:05:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:05:33.304980 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d244a7-0395-4f82-956f-4da74495fc41 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:33.329666 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d244a7-0395-4f82-956f-4da74495fc41 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:33.336528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c10d8d-d78b-41d4-91ba-2bf281687d02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9506139d5744fe69153e7e30a05b436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:33.337263 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c10d8d-d78b-41d4-91ba-2bf281687d02 service placement] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:33.337543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c10d8d-d78b-41d4-91ba-2bf281687d02 service placement] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:33.337839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c10d8d-d78b-41d4-91ba-2bf281687d02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:33.339608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-65c10d8d-d78b-41d4-91ba-2bf281687d02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:33.352280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c10d8d-d78b-41d4-91ba-2bf281687d02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:33.353896 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c10d8d-d78b-41d4-91ba-2bf281687d02 service placement] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:33.366139 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1149/2265] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:33.789728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094750d-a938-4cde-b3f5-10dd8f7e9dda None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:33.791677 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4094750d-a938-4cde-b3f5-10dd8f7e9dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:33.806025 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094750d-a938-4cde-b3f5-10dd8f7e9dda None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:33.809554 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c03f06-efac-44d3-aca3-22e17aa1f8d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:33.810416 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c03f06-efac-44d3-aca3-22e17aa1f8d1 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:33.810696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c03f06-efac-44d3-aca3-22e17aa1f8d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:33.811007 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c03f06-efac-44d3-aca3-22e17aa1f8d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:33.812795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b4c03f06-efac-44d3-aca3-22e17aa1f8d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:33.825402 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c03f06-efac-44d3-aca3-22e17aa1f8d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:33.827199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c03f06-efac-44d3-aca3-22e17aa1f8d1 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:33.841563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1117/2266] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:37.951400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c10d8d-d78b-41d4-91ba-2bf281687d02 service placement] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:37.953001 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-65c10d8d-d78b-41d4-91ba-2bf281687d02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:38.022830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c10d8d-d78b-41d4-91ba-2bf281687d02 service placement] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:38.027763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aa865-62be-4075-b046-9f9c9f54f6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:38.028461 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aa865-62be-4075-b046-9f9c9f54f6e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:38.028775 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aa865-62be-4075-b046-9f9c9f54f6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:38.028999 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aa865-62be-4075-b046-9f9c9f54f6e3 admin admin] PATH_INFO: `/v3/users/29c2921e57dc44d2b6d334e88baae0cd` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:38.030116 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589aa865-62be-4075-b046-9f9c9f54f6e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29c2921e57dc44d2b6d334e88baae0cd)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:38.032156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589aa865-62be-4075-b046-9f9c9f54f6e3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:38.275063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-589aa865-62be-4075-b046-9f9c9f54f6e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29c2921e57dc44d2b6d334e88baae0cd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:38.352216 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-589aa865-62be-4075-b046-9f9c9f54f6e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29c2921e57dc44d2b6d334e88baae0cd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:38.355543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-589aa865-62be-4075-b046-9f9c9f54f6e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29c2921e57dc44d2b6d334e88baae0cd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:38.360919 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:38.363355 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-589aa865-62be-4075-b046-9f9c9f54f6e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29c2921e57dc44d2b6d334e88baae0cd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:38.378840 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-589aa865-62be-4075-b046-9f9c9f54f6e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29c2921e57dc44d2b6d334e88baae0cd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:38.387727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1150/2267] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:05:37 2022] DELETE /identity/v3/users/29c2921e57dc44d2b6d334e88baae0cd => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:05:38.765057 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c03f06-efac-44d3-aca3-22e17aa1f8d1 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:38.779209 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c03f06-efac-44d3-aca3-22e17aa1f8d1 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:38.857612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ac670-0e8e-4b90-b878-f85c551eb2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:38.858538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ac670-0e8e-4b90-b878-f85c551eb2e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:38.858812 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ac670-0e8e-4b90-b878-f85c551eb2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:38.859107 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ac670-0e8e-4b90-b878-f85c551eb2e9 admin admin] PATH_INFO: `/v3/projects/c533fdd2bf51465792fc266cefeb58f7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:38.860292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ac670-0e8e-4b90-b878-f85c551eb2e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c533fdd2bf51465792fc266cefeb58f7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:38.862446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ac670-0e8e-4b90-b878-f85c551eb2e9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:39.044617 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c76ac670-0e8e-4b90-b878-f85c551eb2e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c533fdd2bf51465792fc266cefeb58f7', 'request_id': 'req-c76ac670-0e8e-4b90-b878-f85c551eb2e9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:39.051421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c76ac670-0e8e-4b90-b878-f85c551eb2e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c533fdd2bf51465792fc266cefeb58f7', 'request_id': 'req-c76ac670-0e8e-4b90-b878-f85c551eb2e9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:39.058967 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c76ac670-0e8e-4b90-b878-f85c551eb2e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c533fdd2bf51465792fc266cefeb58f7', 'request_id': 'req-c76ac670-0e8e-4b90-b878-f85c551eb2e9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:39.452189 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c76ac670-0e8e-4b90-b878-f85c551eb2e9 admin admin] The token cache is being invalidate because project c533fdd2bf51465792fc266cefeb58f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:05:39.452978 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c76ac670-0e8e-4b90-b878-f85c551eb2e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:39.453373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c76ac670-0e8e-4b90-b878-f85c551eb2e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:39.461973 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1118/2268] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:05:38 2022] DELETE /identity/v3/projects/c533fdd2bf51465792fc266cefeb58f7 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:05:39.463127 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aa865-62be-4075-b046-9f9c9f54f6e3 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:39.464674 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-589aa865-62be-4075-b046-9f9c9f54f6e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:39.486312 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aa865-62be-4075-b046-9f9c9f54f6e3 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:39.570758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f37a2-8081-4eec-9d8a-9df8db364a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:39.571483 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f37a2-8081-4eec-9d8a-9df8db364a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:39.572007 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f37a2-8081-4eec-9d8a-9df8db364a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:39.572663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f37a2-8081-4eec-9d8a-9df8db364a24 admin admin] PATH_INFO: `/v3/users/f577ab5339c148c29416ebcd4483aa43` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:39.574473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f37a2-8081-4eec-9d8a-9df8db364a24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f577ab5339c148c29416ebcd4483aa43)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:39.576963 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f37a2-8081-4eec-9d8a-9df8db364a24 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:39.783455 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b46f37a2-8081-4eec-9d8a-9df8db364a24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f577ab5339c148c29416ebcd4483aa43'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:39.877552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b46f37a2-8081-4eec-9d8a-9df8db364a24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f577ab5339c148c29416ebcd4483aa43'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:39.878460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b46f37a2-8081-4eec-9d8a-9df8db364a24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f577ab5339c148c29416ebcd4483aa43'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:39.886232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b46f37a2-8081-4eec-9d8a-9df8db364a24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f577ab5339c148c29416ebcd4483aa43'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:39.901005 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b46f37a2-8081-4eec-9d8a-9df8db364a24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f577ab5339c148c29416ebcd4483aa43'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:39.907954 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1151/2269] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:05:39 2022] DELETE /identity/v3/users/f577ab5339c148c29416ebcd4483aa43 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:05:40.227306 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ac670-0e8e-4b90-b878-f85c551eb2e9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:40.228933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c76ac670-0e8e-4b90-b878-f85c551eb2e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:40.244882 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ac670-0e8e-4b90-b878-f85c551eb2e9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:40.262880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:40.312817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e4407-7199-4f6c-b298-26a726e23f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:40.313568 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e4407-7199-4f6c-b298-26a726e23f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:40.313873 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e4407-7199-4f6c-b298-26a726e23f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:40.314361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e4407-7199-4f6c-b298-26a726e23f06 admin admin] PATH_INFO: `/v3/projects/54a65d7e2018462da3c8506472250952` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:40.316667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e4407-7199-4f6c-b298-26a726e23f06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54a65d7e2018462da3c8506472250952)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:40.318603 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e4407-7199-4f6c-b298-26a726e23f06 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:40.482259 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a55e4407-7199-4f6c-b298-26a726e23f06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54a65d7e2018462da3c8506472250952', 'request_id': 'req-a55e4407-7199-4f6c-b298-26a726e23f06'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:40.488702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a55e4407-7199-4f6c-b298-26a726e23f06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54a65d7e2018462da3c8506472250952', 'request_id': 'req-a55e4407-7199-4f6c-b298-26a726e23f06'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:40.495519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a55e4407-7199-4f6c-b298-26a726e23f06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54a65d7e2018462da3c8506472250952', 'request_id': 'req-a55e4407-7199-4f6c-b298-26a726e23f06'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:40.677999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a55e4407-7199-4f6c-b298-26a726e23f06 admin admin] The token cache is being invalidate because project 54a65d7e2018462da3c8506472250952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:05:40.678291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a55e4407-7199-4f6c-b298-26a726e23f06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:40.678949 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a55e4407-7199-4f6c-b298-26a726e23f06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:40.680496 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1119/2270] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:05:40 2022] DELETE /identity/v3/projects/54a65d7e2018462da3c8506472250952 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:05:40.685420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0acfd-ab6c-4ea5-b949-57362a7df846 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:40.685859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0acfd-ab6c-4ea5-b949-57362a7df846 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:40.685964 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0acfd-ab6c-4ea5-b949-57362a7df846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:40.708864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-39b0acfd-ab6c-4ea5-b949-57362a7df846 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:40.710118 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-39b0acfd-ab6c-4ea5-b949-57362a7df846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:40.774851 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:40.785354 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1152/2271] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:05:40.796455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cda99-9679-4ba0-b693-718970a02260 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:40.796455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cda99-9679-4ba0-b693-718970a02260 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:40.796455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cda99-9679-4ba0-b693-718970a02260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:40.816709 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-bb3cda99-9679-4ba0-b693-718970a02260 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:40.818045 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bb3cda99-9679-4ba0-b693-718970a02260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:40.830163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1120/2272] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:05:40.831280 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0acfd-ab6c-4ea5-b949-57362a7df846 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:40.842267 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0acfd-ab6c-4ea5-b949-57362a7df846 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:40.848118 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2f6b8-a0ad-4d11-af0e-fb71f83f156c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:40.848731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2f6b8-a0ad-4d11-af0e-fb71f83f156c admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:40.849181 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2f6b8-a0ad-4d11-af0e-fb71f83f156c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:40.849698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf2f6b8-a0ad-4d11-af0e-fb71f83f156c admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:40.850360 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2f6b8-a0ad-4d11-af0e-fb71f83f156c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:05:40.850689 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2f6b8-a0ad-4d11-af0e-fb71f83f156c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:40.853176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf2f6b8-a0ad-4d11-af0e-fb71f83f156c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:40.870751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1153/2273] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:05:40.871862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cda99-9679-4ba0-b693-718970a02260 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:40.882477 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cda99-9679-4ba0-b693-718970a02260 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:40.887366 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fb240-704a-4dc9-860c-86d8a0a7202e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:40.887968 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fb240-704a-4dc9-860c-86d8a0a7202e admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:40.888246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fb240-704a-4dc9-860c-86d8a0a7202e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:40.888723 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fb240-704a-4dc9-860c-86d8a0a7202e admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:40.889532 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261fb240-704a-4dc9-860c-86d8a0a7202e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:40.891603 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261fb240-704a-4dc9-860c-86d8a0a7202e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:41.011087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1121/2274] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:05:40 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:05:41.015448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2f6b8-a0ad-4d11-af0e-fb71f83f156c admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:41.026242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf2f6b8-a0ad-4d11-af0e-fb71f83f156c admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:41.083411 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173246d-7c99-480b-9e65-a8e5ee29da42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:41.084006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173246d-7c99-480b-9e65-a8e5ee29da42 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:41.084279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173246d-7c99-480b-9e65-a8e5ee29da42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:41.084566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173246d-7c99-480b-9e65-a8e5ee29da42 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:41.085163 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173246d-7c99-480b-9e65-a8e5ee29da42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:41.087095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173246d-7c99-480b-9e65-a8e5ee29da42 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:41.253341 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1154/2275] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:05:41 2022] POST /identity/v3/users => generated 286 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:05:41.257584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fb240-704a-4dc9-860c-86d8a0a7202e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:41.270530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fb240-704a-4dc9-860c-86d8a0a7202e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:41.273465 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e21eb-5a40-409d-a9e9-fbe174ca099e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:41.274321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e21eb-5a40-409d-a9e9-fbe174ca099e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:41.274599 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e21eb-5a40-409d-a9e9-fbe174ca099e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:41.274933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e21eb-5a40-409d-a9e9-fbe174ca099e admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:41.275572 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e21eb-5a40-409d-a9e9-fbe174ca099e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:41.277192 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047e21eb-5a40-409d-a9e9-fbe174ca099e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:41.288271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1122/2276] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:05:41.292361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173246d-7c99-480b-9e65-a8e5ee29da42 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:41.303205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173246d-7c99-480b-9e65-a8e5ee29da42 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:41.307855 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36d11b-692d-4f22-9356-879018e4e1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:41.308557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36d11b-692d-4f22-9356-879018e4e1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:41.308841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36d11b-692d-4f22-9356-879018e4e1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:41.309118 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36d11b-692d-4f22-9356-879018e4e1c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:41.309764 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36d11b-692d-4f22-9356-879018e4e1c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:41.311405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36d11b-692d-4f22-9356-879018e4e1c0 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:41.323474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1155/2277] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:05:41.327698 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e21eb-5a40-409d-a9e9-fbe174ca099e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:41.338667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e21eb-5a40-409d-a9e9-fbe174ca099e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:41.343115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e97b7-bf2d-4e2f-8196-2a66d207c9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:41.343779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e97b7-bf2d-4e2f-8196-2a66d207c9a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:41.344045 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e97b7-bf2d-4e2f-8196-2a66d207c9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:41.344315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e97b7-bf2d-4e2f-8196-2a66d207c9a9 admin admin] PATH_INFO: `/v3/projects/0d4361bb4c424b41aed6f6afa040df84/users/30cfda5bb8284716935e7670bf44f647/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:41.360191 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e97b7-bf2d-4e2f-8196-2a66d207c9a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d4361bb4c424b41aed6f6afa040df84, user_id=30cfda5bb8284716935e7670bf44f647, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:41.362854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e97b7-bf2d-4e2f-8196-2a66d207c9a9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:41.459543 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1123/2278] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:05:41 2022] PUT /identity/v3/projects/0d4361bb4c424b41aed6f6afa040df84/users/30cfda5bb8284716935e7670bf44f647/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:05:41.465416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d491ef-2a0f-4036-b7b5-08c04218a49d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:41.465703 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d491ef-2a0f-4036-b7b5-08c04218a49d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:41.465976 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d491ef-2a0f-4036-b7b5-08c04218a49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:41.486725 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-12d491ef-2a0f-4036-b7b5-08c04218a49d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:41.488130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-12d491ef-2a0f-4036-b7b5-08c04218a49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:41.547775 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1156/2279] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:05:41.557748 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e97b7-bf2d-4e2f-8196-2a66d207c9a9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:41.559523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-595e97b7-bf2d-4e2f-8196-2a66d207c9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:41.606657 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:41.617302 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e97b7-bf2d-4e2f-8196-2a66d207c9a9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:41.621005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84f23b-69aa-4b3e-8809-de1e88649526 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:41.621971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84f23b-69aa-4b3e-8809-de1e88649526 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:41.621971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84f23b-69aa-4b3e-8809-de1e88649526 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:41.622165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84f23b-69aa-4b3e-8809-de1e88649526 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:41.631884 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84f23b-69aa-4b3e-8809-de1e88649526 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:41.633312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84f23b-69aa-4b3e-8809-de1e88649526 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:41.643163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1124/2280] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:50.222773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80eea03-5e7d-4d09-b12a-7f1d5b1b72c5 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:50.223450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80eea03-5e7d-4d09-b12a-7f1d5b1b72c5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:50.223450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80eea03-5e7d-4d09-b12a-7f1d5b1b72c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:50.276201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a80eea03-5e7d-4d09-b12a-7f1d5b1b72c5 None None] MFA Rules not processed for user `30cfda5bb8284716935e7670bf44f647`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:50.279290 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a80eea03-5e7d-4d09-b12a-7f1d5b1b72c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:50.341144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:50.352189 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1157/2281] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:05:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:05:50.356810 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84f23b-69aa-4b3e-8809-de1e88649526 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:50.369978 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84f23b-69aa-4b3e-8809-de1e88649526 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:50.375379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621e158-9d5d-40f6-be2a-dd901ba8cff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:50.376034 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621e158-9d5d-40f6-be2a-dd901ba8cff3 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:50.376265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621e158-9d5d-40f6-be2a-dd901ba8cff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:50.376538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621e158-9d5d-40f6-be2a-dd901ba8cff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:50.377181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621e158-9d5d-40f6-be2a-dd901ba8cff3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:50.379352 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621e158-9d5d-40f6-be2a-dd901ba8cff3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:50.545971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1125/2282] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:05:50 2022] POST /identity/v3/projects => generated 373 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:05:50.547416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80eea03-5e7d-4d09-b12a-7f1d5b1b72c5 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:50.561651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80eea03-5e7d-4d09-b12a-7f1d5b1b72c5 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:50.641046 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3008a91-3e52-444c-8976-6709258dfb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:50.642090 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3008a91-3e52-444c-8976-6709258dfb88 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:50.642090 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3008a91-3e52-444c-8976-6709258dfb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:50.642325 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3008a91-3e52-444c-8976-6709258dfb88 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:50.643036 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3008a91-3e52-444c-8976-6709258dfb88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:50.645052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3008a91-3e52-444c-8976-6709258dfb88 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:50.781223 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1158/2283] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:05:50 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:05:50.785769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621e158-9d5d-40f6-be2a-dd901ba8cff3 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:50.801185 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621e158-9d5d-40f6-be2a-dd901ba8cff3 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:50.807184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ece76-6287-40ef-b493-93f90f5aad51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:50.809026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ece76-6287-40ef-b493-93f90f5aad51 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:50.809667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ece76-6287-40ef-b493-93f90f5aad51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:50.810612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ece76-6287-40ef-b493-93f90f5aad51 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:50.811747 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ece76-6287-40ef-b493-93f90f5aad51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:50.816064 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ece76-6287-40ef-b493-93f90f5aad51 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:50.829048 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1126/2284] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:05:50.839545 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3008a91-3e52-444c-8976-6709258dfb88 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:50.851169 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3008a91-3e52-444c-8976-6709258dfb88 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:50.859147 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fcd82-1752-4cfe-ace3-c2bf984ab2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:50.859905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fcd82-1752-4cfe-ace3-c2bf984ab2f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:50.860274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fcd82-1752-4cfe-ace3-c2bf984ab2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:50.860580 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fcd82-1752-4cfe-ace3-c2bf984ab2f6 admin admin] PATH_INFO: `/v3/projects/b20af97c0ebd4b0684d8af91ff968997/users/dc7727d9b8864d7c9eb7f3ba02f245f2/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:50.900051 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445fcd82-1752-4cfe-ace3-c2bf984ab2f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b20af97c0ebd4b0684d8af91ff968997, user_id=dc7727d9b8864d7c9eb7f3ba02f245f2, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:50.900051 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445fcd82-1752-4cfe-ace3-c2bf984ab2f6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:51.056175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1159/2285] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:05:50 2022] PUT /identity/v3/projects/b20af97c0ebd4b0684d8af91ff968997/users/dc7727d9b8864d7c9eb7f3ba02f245f2/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:05:51.060626 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ece76-6287-40ef-b493-93f90f5aad51 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:51.072302 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ece76-6287-40ef-b493-93f90f5aad51 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:51.143860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0716f2d-bc64-4aae-9952-622f0d48187d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:51.144938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0716f2d-bc64-4aae-9952-622f0d48187d admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:51.145241 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0716f2d-bc64-4aae-9952-622f0d48187d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:51.145521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0716f2d-bc64-4aae-9952-622f0d48187d admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:51.147014 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0716f2d-bc64-4aae-9952-622f0d48187d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:51.150122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0716f2d-bc64-4aae-9952-622f0d48187d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:51.166383 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1127/2286] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:05:51.171742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fcd82-1752-4cfe-ace3-c2bf984ab2f6 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:51.182693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fcd82-1752-4cfe-ace3-c2bf984ab2f6 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:51.190208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3fefa-7303-4f48-93bb-5ab94f74f554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:51.190911 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3fefa-7303-4f48-93bb-5ab94f74f554 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:51.191345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3fefa-7303-4f48-93bb-5ab94f74f554 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:51.191682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3fefa-7303-4f48-93bb-5ab94f74f554 admin admin] PATH_INFO: `/v3/projects/b20af97c0ebd4b0684d8af91ff968997/users/dc7727d9b8864d7c9eb7f3ba02f245f2/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:51.193802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3fefa-7303-4f48-93bb-5ab94f74f554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b20af97c0ebd4b0684d8af91ff968997, user_id=dc7727d9b8864d7c9eb7f3ba02f245f2, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:51.196584 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3fefa-7303-4f48-93bb-5ab94f74f554 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:51.207419 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1160/2287] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:05:51 2022] PUT /identity/v3/projects/b20af97c0ebd4b0684d8af91ff968997/users/dc7727d9b8864d7c9eb7f3ba02f245f2/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:05:53.032859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d0b98-84f6-4b46-8471-9b006e892902 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:53.033421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d0b98-84f6-4b46-8471-9b006e892902 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:53.033421 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d0b98-84f6-4b46-8471-9b006e892902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:53.057014 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d03d0b98-84f6-4b46-8471-9b006e892902 None None] MFA Rules not processed for user `beb41eb66e8140a6999b0bddab3c4afd`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:53.058497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d03d0b98-84f6-4b46-8471-9b006e892902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:53.170523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1128/2288] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:05:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:05:58.937029 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596747f3-601a-453b-bb56-5102b4116806 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:58.937665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596747f3-601a-453b-bb56-5102b4116806 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:58.937665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596747f3-601a-453b-bb56-5102b4116806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:58.977779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-596747f3-601a-453b-bb56-5102b4116806 None None] MFA Rules not processed for user `dc7727d9b8864d7c9eb7f3ba02f245f2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:58.979320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-596747f3-601a-453b-bb56-5102b4116806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:59.081141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1161/2289] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:05:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:05:59.095652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d0b98-84f6-4b46-8471-9b006e892902 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:59.097363 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d03d0b98-84f6-4b46-8471-9b006e892902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:59.191043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:59.212880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d0b98-84f6-4b46-8471-9b006e892902 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:59.219575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98721a03-9e94-49c3-9811-0486bb50dd3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:59.220196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98721a03-9e94-49c3-9811-0486bb50dd3d service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:59.220472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98721a03-9e94-49c3-9811-0486bb50dd3d service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:59.220743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98721a03-9e94-49c3-9811-0486bb50dd3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:59.280366 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98721a03-9e94-49c3-9811-0486bb50dd3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:59.281979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98721a03-9e94-49c3-9811-0486bb50dd3d service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:59.292175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1129/2290] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:03.201865 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28f495-1608-4ecc-b38e-025c7bb8d7e7 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:03.204151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28f495-1608-4ecc-b38e-025c7bb8d7e7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:03.204151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28f495-1608-4ecc-b38e-025c7bb8d7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:03.222672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-fe28f495-1608-4ecc-b38e-025c7bb8d7e7 None None] MFA Rules not processed for user `30cfda5bb8284716935e7670bf44f647`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:03.223955 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fe28f495-1608-4ecc-b38e-025c7bb8d7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:03.231591 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1162/2291] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:03.855582 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98721a03-9e94-49c3-9811-0486bb50dd3d service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:03.858609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-98721a03-9e94-49c3-9811-0486bb50dd3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:03.941478 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98721a03-9e94-49c3-9811-0486bb50dd3d service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:03.945915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4e048-345e-499a-b701-c62acb246691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:03.946749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4e048-345e-499a-b701-c62acb246691 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:03.947362 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4e048-345e-499a-b701-c62acb246691 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:03.947634 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4e048-345e-499a-b701-c62acb246691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:03.949516 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-00d4e048-345e-499a-b701-c62acb246691 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:04.025309 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4e048-345e-499a-b701-c62acb246691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:04.028333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4e048-345e-499a-b701-c62acb246691 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:04.039802 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1130/2292] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:06:08.622991 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28f495-1608-4ecc-b38e-025c7bb8d7e7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:08.624936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fe28f495-1608-4ecc-b38e-025c7bb8d7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:08.667912 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:08.736105 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28f495-1608-4ecc-b38e-025c7bb8d7e7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:08.741360 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12928b-eaba-48a0-ba94-cbfc2f7aa9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:08.742196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12928b-eaba-48a0-ba94-cbfc2f7aa9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:08.742702 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12928b-eaba-48a0-ba94-cbfc2f7aa9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:08.743008 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12928b-eaba-48a0-ba94-cbfc2f7aa9ae admin admin] PATH_INFO: `/v3/users/e4916206c2ff42669276e0a93d6483ab` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:08.744284 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12928b-eaba-48a0-ba94-cbfc2f7aa9ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4916206c2ff42669276e0a93d6483ab)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:08.746251 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12928b-eaba-48a0-ba94-cbfc2f7aa9ae admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:08.750311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4e048-345e-499a-b701-c62acb246691 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:08.806109 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4e048-345e-499a-b701-c62acb246691 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:08.812207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553809c1-054c-46d9-81a2-9d4f28c0ebf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:08.812910 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553809c1-054c-46d9-81a2-9d4f28c0ebf0 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:08.813211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553809c1-054c-46d9-81a2-9d4f28c0ebf0 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:08.813474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553809c1-054c-46d9-81a2-9d4f28c0ebf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:08.815833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-553809c1-054c-46d9-81a2-9d4f28c0ebf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:08.823970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:08.841044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553809c1-054c-46d9-81a2-9d4f28c0ebf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:08.842941 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553809c1-054c-46d9-81a2-9d4f28c0ebf0 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:08.857834 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1131/2293] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:06:09.024630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ab12928b-eaba-48a0-ba94-cbfc2f7aa9ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4916206c2ff42669276e0a93d6483ab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:09.118272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ab12928b-eaba-48a0-ba94-cbfc2f7aa9ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4916206c2ff42669276e0a93d6483ab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:09.118635 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ab12928b-eaba-48a0-ba94-cbfc2f7aa9ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4916206c2ff42669276e0a93d6483ab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:09.127297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ab12928b-eaba-48a0-ba94-cbfc2f7aa9ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4916206c2ff42669276e0a93d6483ab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:09.143973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ab12928b-eaba-48a0-ba94-cbfc2f7aa9ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4916206c2ff42669276e0a93d6483ab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:09.150832 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1163/2294] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:06:08 2022] DELETE /identity/v3/users/e4916206c2ff42669276e0a93d6483ab => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:09.630659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553809c1-054c-46d9-81a2-9d4f28c0ebf0 service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:09.636453 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553809c1-054c-46d9-81a2-9d4f28c0ebf0 service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:09.701648 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ba850-5fc8-4541-beee-68e6e3c4d8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:09.702202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ba850-5fc8-4541-beee-68e6e3c4d8b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:09.702480 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ba850-5fc8-4541-beee-68e6e3c4d8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:09.702749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ba850-5fc8-4541-beee-68e6e3c4d8b0 admin admin] PATH_INFO: `/v3/projects/2078beb45799497ea367c552b6d8bcf9` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:09.703915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ba850-5fc8-4541-beee-68e6e3c4d8b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2078beb45799497ea367c552b6d8bcf9)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:09.705852 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ba850-5fc8-4541-beee-68e6e3c4d8b0 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:09.839549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-945ba850-5fc8-4541-beee-68e6e3c4d8b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2078beb45799497ea367c552b6d8bcf9', 'request_id': 'req-945ba850-5fc8-4541-beee-68e6e3c4d8b0'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:09.847046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-945ba850-5fc8-4541-beee-68e6e3c4d8b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2078beb45799497ea367c552b6d8bcf9', 'request_id': 'req-945ba850-5fc8-4541-beee-68e6e3c4d8b0'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:09.854692 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-945ba850-5fc8-4541-beee-68e6e3c4d8b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2078beb45799497ea367c552b6d8bcf9', 'request_id': 'req-945ba850-5fc8-4541-beee-68e6e3c4d8b0'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:09.984026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-945ba850-5fc8-4541-beee-68e6e3c4d8b0 admin admin] The token cache is being invalidate because project 2078beb45799497ea367c552b6d8bcf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:06:09.984229 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-945ba850-5fc8-4541-beee-68e6e3c4d8b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:09.984854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-945ba850-5fc8-4541-beee-68e6e3c4d8b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:09.986374 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1132/2295] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:06:09 2022] DELETE /identity/v3/projects/2078beb45799497ea367c552b6d8bcf9 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:09.992502 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223d933-9977-4c11-b8fe-7f8a8c5b8a67 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:09.993032 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223d933-9977-4c11-b8fe-7f8a8c5b8a67 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:09.993032 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223d933-9977-4c11-b8fe-7f8a8c5b8a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:10.034709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-4223d933-9977-4c11-b8fe-7f8a8c5b8a67 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:10.036457 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4223d933-9977-4c11-b8fe-7f8a8c5b8a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:10.099323 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1164/2296] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:10.107327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acf3de-c139-4541-b50d-db4b721f7cad None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:10.107616 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acf3de-c139-4541-b50d-db4b721f7cad None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:10.107893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acf3de-c139-4541-b50d-db4b721f7cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:10.129900 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b3acf3de-c139-4541-b50d-db4b721f7cad None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:10.131207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b3acf3de-c139-4541-b50d-db4b721f7cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:10.139615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1133/2297] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:10.145261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223d933-9977-4c11-b8fe-7f8a8c5b8a67 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:10.156841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223d933-9977-4c11-b8fe-7f8a8c5b8a67 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:10.163371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c6b09-e283-4e3f-a10e-315cc0b65c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:10.164473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c6b09-e283-4e3f-a10e-315cc0b65c35 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:10.164473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c6b09-e283-4e3f-a10e-315cc0b65c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:10.164769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c6b09-e283-4e3f-a10e-315cc0b65c35 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:10.165506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c6b09-e283-4e3f-a10e-315cc0b65c35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:06:10.165855 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c6b09-e283-4e3f-a10e-315cc0b65c35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:10.167271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c6b09-e283-4e3f-a10e-315cc0b65c35 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:10.184376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1165/2298] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:06:10.192166 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acf3de-c139-4541-b50d-db4b721f7cad None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:10.202559 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acf3de-c139-4541-b50d-db4b721f7cad None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:10.208229 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f8b6c-d738-4b0c-b283-f2a77fa73290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:10.208879 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f8b6c-d738-4b0c-b283-f2a77fa73290 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:10.209170 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f8b6c-d738-4b0c-b283-f2a77fa73290 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:10.209447 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f8b6c-d738-4b0c-b283-f2a77fa73290 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:10.210132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f8b6c-d738-4b0c-b283-f2a77fa73290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:10.212126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f8b6c-d738-4b0c-b283-f2a77fa73290 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:10.466699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1134/2299] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:06:10 2022] POST /identity/v3/projects => generated 369 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:06:10.471065 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c6b09-e283-4e3f-a10e-315cc0b65c35 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:10.488362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c6b09-e283-4e3f-a10e-315cc0b65c35 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:10.492570 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:10.558529 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3eb7d-cf2d-4c82-8929-73d4c3c94387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:10.559095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3eb7d-cf2d-4c82-8929-73d4c3c94387 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:10.559413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3eb7d-cf2d-4c82-8929-73d4c3c94387 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:10.559923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3eb7d-cf2d-4c82-8929-73d4c3c94387 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:10.560575 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3eb7d-cf2d-4c82-8929-73d4c3c94387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:10.562524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3eb7d-cf2d-4c82-8929-73d4c3c94387 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:10.705190 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1166/2300] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:06:10 2022] POST /identity/v3/users => generated 285 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:06:10.709455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f8b6c-d738-4b0c-b283-f2a77fa73290 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:10.720417 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f8b6c-d738-4b0c-b283-f2a77fa73290 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:10.725242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb80ce0-f133-4563-a003-69bd6281a489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:10.725907 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb80ce0-f133-4563-a003-69bd6281a489 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:10.726188 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb80ce0-f133-4563-a003-69bd6281a489 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:10.726462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb80ce0-f133-4563-a003-69bd6281a489 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:10.727269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb80ce0-f133-4563-a003-69bd6281a489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:10.728846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb80ce0-f133-4563-a003-69bd6281a489 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:10.741997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1135/2301] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:10.744559 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3eb7d-cf2d-4c82-8929-73d4c3c94387 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:10.755925 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3eb7d-cf2d-4c82-8929-73d4c3c94387 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:10.760788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b1b17-423b-4f3d-8d58-e9512e82f420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:10.761436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b1b17-423b-4f3d-8d58-e9512e82f420 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:10.761710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b1b17-423b-4f3d-8d58-e9512e82f420 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:10.762010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b1b17-423b-4f3d-8d58-e9512e82f420 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:10.762668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b1b17-423b-4f3d-8d58-e9512e82f420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:10.764477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b1b17-423b-4f3d-8d58-e9512e82f420 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:10.778050 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1167/2302] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:10.782286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb80ce0-f133-4563-a003-69bd6281a489 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:10.792944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb80ce0-f133-4563-a003-69bd6281a489 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:10.798008 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7109e-80fe-4318-bcbe-c3127cd6245d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:10.798655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7109e-80fe-4318-bcbe-c3127cd6245d admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:10.799096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7109e-80fe-4318-bcbe-c3127cd6245d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:10.799371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7109e-80fe-4318-bcbe-c3127cd6245d admin admin] PATH_INFO: `/v3/projects/698078c4bc7d475a8206730acc29e4c5/users/2c95e11bc1204ab1b3e531c065d2c4d5/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:10.817929 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7109e-80fe-4318-bcbe-c3127cd6245d admin admin] RBAC: Authorizing `identity:create_grant(project_id=698078c4bc7d475a8206730acc29e4c5, user_id=2c95e11bc1204ab1b3e531c065d2c4d5, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:10.820561 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7109e-80fe-4318-bcbe-c3127cd6245d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:10.923900 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1136/2303] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:06:10 2022] PUT /identity/v3/projects/698078c4bc7d475a8206730acc29e4c5/users/2c95e11bc1204ab1b3e531c065d2c4d5/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:10.935629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c80a5c-360c-4c81-b796-59e83749da9a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:10.936033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c80a5c-360c-4c81-b796-59e83749da9a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:10.936420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c80a5c-360c-4c81-b796-59e83749da9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:10.967630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-64c80a5c-360c-4c81-b796-59e83749da9a None None] MFA Rules not processed for user `2c95e11bc1204ab1b3e531c065d2c4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:10.968817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-64c80a5c-360c-4c81-b796-59e83749da9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:11.043218 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1168/2304] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:11.048054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7109e-80fe-4318-bcbe-c3127cd6245d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:11.067205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:11.116486 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7109e-80fe-4318-bcbe-c3127cd6245d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:11.121291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a0a0e-c687-42cc-88f6-fd23a5c67200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:11.121891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a0a0e-c687-42cc-88f6-fd23a5c67200 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:11.122165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a0a0e-c687-42cc-88f6-fd23a5c67200 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:11.122433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a0a0e-c687-42cc-88f6-fd23a5c67200 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:11.123095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a0a0e-c687-42cc-88f6-fd23a5c67200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:11.125256 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a0a0e-c687-42cc-88f6-fd23a5c67200 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:11.227053 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1137/2305] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:06:11 2022] POST /identity/v3/projects => generated 369 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:06:11.234202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c80a5c-360c-4c81-b796-59e83749da9a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:11.245698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c80a5c-360c-4c81-b796-59e83749da9a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:11.306858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd5957-4ba4-47e5-b122-67f1ed0cf213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:11.307507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd5957-4ba4-47e5-b122-67f1ed0cf213 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:11.307791 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd5957-4ba4-47e5-b122-67f1ed0cf213 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:11.308154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd5957-4ba4-47e5-b122-67f1ed0cf213 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:11.308788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd5957-4ba4-47e5-b122-67f1ed0cf213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:11.310772 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd5957-4ba4-47e5-b122-67f1ed0cf213 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:11.399181 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1169/2306] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:06:11 2022] POST /identity/v3/users => generated 291 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:06:11.403358 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a0a0e-c687-42cc-88f6-fd23a5c67200 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:11.414428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a0a0e-c687-42cc-88f6-fd23a5c67200 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:11.419315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397003fd-6790-4319-b4d6-7dea3d31d274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:11.419937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397003fd-6790-4319-b4d6-7dea3d31d274 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:11.420217 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397003fd-6790-4319-b4d6-7dea3d31d274 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:11.420521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397003fd-6790-4319-b4d6-7dea3d31d274 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:11.421169 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397003fd-6790-4319-b4d6-7dea3d31d274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:11.422783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397003fd-6790-4319-b4d6-7dea3d31d274 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:11.434162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1138/2307] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:11.438485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd5957-4ba4-47e5-b122-67f1ed0cf213 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:11.445369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:11.449776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd5957-4ba4-47e5-b122-67f1ed0cf213 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:11.454582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33af19-9273-4473-9184-d930ea2dcc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:11.460000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33af19-9273-4473-9184-d930ea2dcc84 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:11.460000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33af19-9273-4473-9184-d930ea2dcc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:11.460000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33af19-9273-4473-9184-d930ea2dcc84 admin admin] PATH_INFO: `/v3/projects/dae1bbc696f14abaa00767ecbd6e8f7f/users/38695dc6e5364f0392d3a623d94b7370/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:11.471852 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33af19-9273-4473-9184-d930ea2dcc84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae1bbc696f14abaa00767ecbd6e8f7f, user_id=38695dc6e5364f0392d3a623d94b7370, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:11.477271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33af19-9273-4473-9184-d930ea2dcc84 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:11.552935 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1170/2308] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:06:11 2022] PUT /identity/v3/projects/dae1bbc696f14abaa00767ecbd6e8f7f/users/38695dc6e5364f0392d3a623d94b7370/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:11.557365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397003fd-6790-4319-b4d6-7dea3d31d274 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:11.568678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397003fd-6790-4319-b4d6-7dea3d31d274 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:11.632624 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a67bd-135a-4389-83a7-5885418e3274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:11.633124 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a67bd-135a-4389-83a7-5885418e3274 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:11.633396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a67bd-135a-4389-83a7-5885418e3274 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:11.633661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a67bd-135a-4389-83a7-5885418e3274 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:11.634354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a67bd-135a-4389-83a7-5885418e3274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:11.636041 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a67bd-135a-4389-83a7-5885418e3274 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:11.646652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1139/2309] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:11.651527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33af19-9273-4473-9184-d930ea2dcc84 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:11.663665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33af19-9273-4473-9184-d930ea2dcc84 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:11.668431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78fa66-bc43-4445-bbed-9809891890d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:11.669635 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78fa66-bc43-4445-bbed-9809891890d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:11.669635 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78fa66-bc43-4445-bbed-9809891890d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:11.669860 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de78fa66-bc43-4445-bbed-9809891890d3 admin admin] PATH_INFO: `/v3/projects/dae1bbc696f14abaa00767ecbd6e8f7f/users/38695dc6e5364f0392d3a623d94b7370/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:11.671653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de78fa66-bc43-4445-bbed-9809891890d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae1bbc696f14abaa00767ecbd6e8f7f, user_id=38695dc6e5364f0392d3a623d94b7370, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:11.676332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de78fa66-bc43-4445-bbed-9809891890d3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:11.689407 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1171/2310] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:06:11 2022] PUT /identity/v3/projects/dae1bbc696f14abaa00767ecbd6e8f7f/users/38695dc6e5364f0392d3a623d94b7370/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:11.700074 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816dbf2-1018-49d4-8539-d73ae5ba0dca None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:11.705525 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816dbf2-1018-49d4-8539-d73ae5ba0dca None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:11.705525 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816dbf2-1018-49d4-8539-d73ae5ba0dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:11.732783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b816dbf2-1018-49d4-8539-d73ae5ba0dca None None] MFA Rules not processed for user `38695dc6e5364f0392d3a623d94b7370`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:11.734128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b816dbf2-1018-49d4-8539-d73ae5ba0dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:11.823272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1140/2311] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:11.835376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78fa66-bc43-4445-bbed-9809891890d3 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:11.837421 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-de78fa66-bc43-4445-bbed-9809891890d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:11.928774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de78fa66-bc43-4445-bbed-9809891890d3 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:11.939603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b66a4-eb4f-4fb9-8c2b-83905990cf36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:11.940219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b66a4-eb4f-4fb9-8c2b-83905990cf36 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:11.940495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b66a4-eb4f-4fb9-8c2b-83905990cf36 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:11.940794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b66a4-eb4f-4fb9-8c2b-83905990cf36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:11.951302 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b66a4-eb4f-4fb9-8c2b-83905990cf36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:11.952854 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b66a4-eb4f-4fb9-8c2b-83905990cf36 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:11.964376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1172/2312] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:15.782599 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816dbf2-1018-49d4-8539-d73ae5ba0dca None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:15.796863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:15.858698 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816dbf2-1018-49d4-8539-d73ae5ba0dca None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:15.864573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9016eee-b3df-4806-9e2c-8606e8e565b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:15.869251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9016eee-b3df-4806-9e2c-8606e8e565b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:15.869251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9016eee-b3df-4806-9e2c-8606e8e565b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:15.869251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9016eee-b3df-4806-9e2c-8606e8e565b5 admin admin] PATH_INFO: `/v3/users/2c95e11bc1204ab1b3e531c065d2c4d5` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:15.869251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9016eee-b3df-4806-9e2c-8606e8e565b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c95e11bc1204ab1b3e531c065d2c4d5)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:15.869251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9016eee-b3df-4806-9e2c-8606e8e565b5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:16.086312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b9016eee-b3df-4806-9e2c-8606e8e565b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c95e11bc1204ab1b3e531c065d2c4d5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:16.178547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b9016eee-b3df-4806-9e2c-8606e8e565b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c95e11bc1204ab1b3e531c065d2c4d5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:16.191489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b9016eee-b3df-4806-9e2c-8606e8e565b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c95e11bc1204ab1b3e531c065d2c4d5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:16.192132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b9016eee-b3df-4806-9e2c-8606e8e565b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c95e11bc1204ab1b3e531c065d2c4d5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:16.199064 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b9016eee-b3df-4806-9e2c-8606e8e565b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c95e11bc1204ab1b3e531c065d2c4d5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:16.205407 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1141/2313] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:06:15 2022] DELETE /identity/v3/users/2c95e11bc1204ab1b3e531c065d2c4d5 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:16.209938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e1743-13d6-4484-8bc0-38164c1c1eba None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:16.210415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e1743-13d6-4484-8bc0-38164c1c1eba None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:16.210527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e1743-13d6-4484-8bc0-38164c1c1eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:16.230059 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a60e1743-13d6-4484-8bc0-38164c1c1eba None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:16.231293 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a60e1743-13d6-4484-8bc0-38164c1c1eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:16.289386 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1173/2314] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:16.299446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9016eee-b3df-4806-9e2c-8606e8e565b5 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:16.305314 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b9016eee-b3df-4806-9e2c-8606e8e565b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:16.382692 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9016eee-b3df-4806-9e2c-8606e8e565b5 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:16.386387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09be249-d6e7-46c4-b70c-b323e27f4f9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:16.387020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09be249-d6e7-46c4-b70c-b323e27f4f9f service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:16.387299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09be249-d6e7-46c4-b70c-b323e27f4f9f service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:16.387570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09be249-d6e7-46c4-b70c-b323e27f4f9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:16.397110 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09be249-d6e7-46c4-b70c-b323e27f4f9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:16.398846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09be249-d6e7-46c4-b70c-b323e27f4f9f service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:16.408295 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1142/2315] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:17.113223 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e1743-13d6-4484-8bc0-38164c1c1eba None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:17.115260 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a60e1743-13d6-4484-8bc0-38164c1c1eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:17.129634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e1743-13d6-4484-8bc0-38164c1c1eba None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:17.135153 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6727da-4f66-4780-b718-1577bc9d9e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:17.136005 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6727da-4f66-4780-b718-1577bc9d9e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:17.136297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6727da-4f66-4780-b718-1577bc9d9e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:17.136572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6727da-4f66-4780-b718-1577bc9d9e78 admin admin] PATH_INFO: `/v3/projects/698078c4bc7d475a8206730acc29e4c5` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:17.137752 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6727da-4f66-4780-b718-1577bc9d9e78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=698078c4bc7d475a8206730acc29e4c5)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:17.139878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6727da-4f66-4780-b718-1577bc9d9e78 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:17.279743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:17.282807 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2a6727da-4f66-4780-b718-1577bc9d9e78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '698078c4bc7d475a8206730acc29e4c5', 'request_id': 'req-2a6727da-4f66-4780-b718-1577bc9d9e78'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:17.357691 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2a6727da-4f66-4780-b718-1577bc9d9e78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '698078c4bc7d475a8206730acc29e4c5', 'request_id': 'req-2a6727da-4f66-4780-b718-1577bc9d9e78'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:17.365696 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2a6727da-4f66-4780-b718-1577bc9d9e78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '698078c4bc7d475a8206730acc29e4c5', 'request_id': 'req-2a6727da-4f66-4780-b718-1577bc9d9e78'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:17.381786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2a6727da-4f66-4780-b718-1577bc9d9e78 admin admin] The token cache is being invalidate because project 698078c4bc7d475a8206730acc29e4c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:06:17.381786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2a6727da-4f66-4780-b718-1577bc9d9e78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:17.381786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2a6727da-4f66-4780-b718-1577bc9d9e78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:17.381786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1174/2316] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:06:17 2022] DELETE /identity/v3/projects/698078c4bc7d475a8206730acc29e4c5 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:17.384615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09be249-d6e7-46c4-b70c-b323e27f4f9f service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:17.395579 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09be249-d6e7-46c4-b70c-b323e27f4f9f service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:17.405581 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:17.456166 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca5cd3-64f0-4cbe-9384-bbe2cceea40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:17.456804 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca5cd3-64f0-4cbe-9384-bbe2cceea40a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:17.457080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca5cd3-64f0-4cbe-9384-bbe2cceea40a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:17.457355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca5cd3-64f0-4cbe-9384-bbe2cceea40a admin admin] PATH_INFO: `/v3/users/38695dc6e5364f0392d3a623d94b7370` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:17.458525 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca5cd3-64f0-4cbe-9384-bbe2cceea40a admin admin] RBAC: Authorizing `identity:delete_user(user_id=38695dc6e5364f0392d3a623d94b7370)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:17.460483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca5cd3-64f0-4cbe-9384-bbe2cceea40a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:17.683717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d3ca5cd3-64f0-4cbe-9384-bbe2cceea40a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38695dc6e5364f0392d3a623d94b7370'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:17.773053 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d3ca5cd3-64f0-4cbe-9384-bbe2cceea40a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38695dc6e5364f0392d3a623d94b7370'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:17.789194 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d3ca5cd3-64f0-4cbe-9384-bbe2cceea40a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38695dc6e5364f0392d3a623d94b7370'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:17.795399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d3ca5cd3-64f0-4cbe-9384-bbe2cceea40a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38695dc6e5364f0392d3a623d94b7370'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:17.803844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d3ca5cd3-64f0-4cbe-9384-bbe2cceea40a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38695dc6e5364f0392d3a623d94b7370'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:17.810725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1143/2317] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:06:17 2022] DELETE /identity/v3/users/38695dc6e5364f0392d3a623d94b7370 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:18.336139 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6727da-4f66-4780-b718-1577bc9d9e78 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:18.337643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2a6727da-4f66-4780-b718-1577bc9d9e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:18.351772 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6727da-4f66-4780-b718-1577bc9d9e78 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:18.415014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09e9ad-0fe2-4027-b0ea-17765859ad8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:18.415659 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09e9ad-0fe2-4027-b0ea-17765859ad8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:18.416024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09e9ad-0fe2-4027-b0ea-17765859ad8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:18.416320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09e9ad-0fe2-4027-b0ea-17765859ad8d admin admin] PATH_INFO: `/v3/projects/dae1bbc696f14abaa00767ecbd6e8f7f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:18.417458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09e9ad-0fe2-4027-b0ea-17765859ad8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae1bbc696f14abaa00767ecbd6e8f7f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:18.419440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d09e9ad-0fe2-4027-b0ea-17765859ad8d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:18.608429 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5d09e9ad-0fe2-4027-b0ea-17765859ad8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae1bbc696f14abaa00767ecbd6e8f7f', 'request_id': 'req-5d09e9ad-0fe2-4027-b0ea-17765859ad8d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:18.706400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5d09e9ad-0fe2-4027-b0ea-17765859ad8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae1bbc696f14abaa00767ecbd6e8f7f', 'request_id': 'req-5d09e9ad-0fe2-4027-b0ea-17765859ad8d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:18.714575 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5d09e9ad-0fe2-4027-b0ea-17765859ad8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae1bbc696f14abaa00767ecbd6e8f7f', 'request_id': 'req-5d09e9ad-0fe2-4027-b0ea-17765859ad8d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:18.726433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5d09e9ad-0fe2-4027-b0ea-17765859ad8d admin admin] The token cache is being invalidate because project dae1bbc696f14abaa00767ecbd6e8f7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:06:18.726724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5d09e9ad-0fe2-4027-b0ea-17765859ad8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:18.727420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5d09e9ad-0fe2-4027-b0ea-17765859ad8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:18.728947 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1175/2318] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:06:18 2022] DELETE /identity/v3/projects/dae1bbc696f14abaa00767ecbd6e8f7f => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:18.734186 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc48a7-f514-479d-849e-491d45d7cd61 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:18.734721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc48a7-f514-479d-849e-491d45d7cd61 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:18.734721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc48a7-f514-479d-849e-491d45d7cd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:18.755146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-53dc48a7-f514-479d-849e-491d45d7cd61 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:18.756393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-53dc48a7-f514-479d-849e-491d45d7cd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:18.841105 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1144/2319] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:18.851048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07811d58-642b-4556-9337-70ae4a1e1cd9 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:18.851346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07811d58-642b-4556-9337-70ae4a1e1cd9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:18.851640 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07811d58-642b-4556-9337-70ae4a1e1cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:18.873021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-07811d58-642b-4556-9337-70ae4a1e1cd9 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:18.874213 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-07811d58-642b-4556-9337-70ae4a1e1cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:18.882412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1176/2320] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:18.887183 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc48a7-f514-479d-849e-491d45d7cd61 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:18.898940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc48a7-f514-479d-849e-491d45d7cd61 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:18.904006 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d86d3-54f7-4046-88b3-c60bb0ffdecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:18.904650 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d86d3-54f7-4046-88b3-c60bb0ffdecc admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:18.904923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d86d3-54f7-4046-88b3-c60bb0ffdecc admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:18.905210 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d86d3-54f7-4046-88b3-c60bb0ffdecc admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:18.905845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d86d3-54f7-4046-88b3-c60bb0ffdecc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:06:18.906143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d86d3-54f7-4046-88b3-c60bb0ffdecc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:18.908004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d86d3-54f7-4046-88b3-c60bb0ffdecc admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:18.928589 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1145/2321] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:06:18.933142 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07811d58-642b-4556-9337-70ae4a1e1cd9 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:18.942841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07811d58-642b-4556-9337-70ae4a1e1cd9 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:18.947677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a7a5b-84b4-4ca4-9953-ba7e40234042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:18.948314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a7a5b-84b4-4ca4-9953-ba7e40234042 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:18.948598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a7a5b-84b4-4ca4-9953-ba7e40234042 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:18.948871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a7a5b-84b4-4ca4-9953-ba7e40234042 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:18.949547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a7a5b-84b4-4ca4-9953-ba7e40234042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:18.951546 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a7a5b-84b4-4ca4-9953-ba7e40234042 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:19.062441 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1177/2322] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:06:18 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:06:19.066961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d86d3-54f7-4046-88b3-c60bb0ffdecc admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:19.078103 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d86d3-54f7-4046-88b3-c60bb0ffdecc admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:19.097187 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:19.139028 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a1f8b-951b-40e4-93b5-7ecef50d6bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:19.139557 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a1f8b-951b-40e4-93b5-7ecef50d6bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:19.139980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a1f8b-951b-40e4-93b5-7ecef50d6bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:19.140261 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a1f8b-951b-40e4-93b5-7ecef50d6bb9 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:19.141933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a1f8b-951b-40e4-93b5-7ecef50d6bb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:19.143072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a1f8b-951b-40e4-93b5-7ecef50d6bb9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:19.339974 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1146/2323] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:06:19 2022] POST /identity/v3/users => generated 291 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:06:19.344786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a7a5b-84b4-4ca4-9953-ba7e40234042 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:19.355839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a7a5b-84b4-4ca4-9953-ba7e40234042 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:19.360453 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a8653-d843-457b-a63c-c399b2879689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:19.361056 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a8653-d843-457b-a63c-c399b2879689 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:19.361335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a8653-d843-457b-a63c-c399b2879689 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:19.361608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a8653-d843-457b-a63c-c399b2879689 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:19.362248 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a8653-d843-457b-a63c-c399b2879689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:19.363934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a8653-d843-457b-a63c-c399b2879689 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:19.375937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1178/2324] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:19.380551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a1f8b-951b-40e4-93b5-7ecef50d6bb9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:19.392418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a1f8b-951b-40e4-93b5-7ecef50d6bb9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:19.397218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8d3cb-cc52-49dc-80bd-63bc5b9d90fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:19.397832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8d3cb-cc52-49dc-80bd-63bc5b9d90fd admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:19.398111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8d3cb-cc52-49dc-80bd-63bc5b9d90fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:19.398386 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8d3cb-cc52-49dc-80bd-63bc5b9d90fd admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:19.399325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8d3cb-cc52-49dc-80bd-63bc5b9d90fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:19.400929 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8d3cb-cc52-49dc-80bd-63bc5b9d90fd admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:19.412346 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1147/2325] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:19.416942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a8653-d843-457b-a63c-c399b2879689 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:19.430784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a8653-d843-457b-a63c-c399b2879689 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:19.435829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98368e31-b556-4f21-be5d-c5506d7f3562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:19.436614 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98368e31-b556-4f21-be5d-c5506d7f3562 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:19.436897 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98368e31-b556-4f21-be5d-c5506d7f3562 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:19.437176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98368e31-b556-4f21-be5d-c5506d7f3562 admin admin] PATH_INFO: `/v3/projects/82ed179e84e84e219b9581e87f6da3fc/users/1d9fe50ddcbc4ed1bfec2d087be521a7/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:19.454950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98368e31-b556-4f21-be5d-c5506d7f3562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ed179e84e84e219b9581e87f6da3fc, user_id=1d9fe50ddcbc4ed1bfec2d087be521a7, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:19.457816 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98368e31-b556-4f21-be5d-c5506d7f3562 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:19.551302 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:19.557276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1179/2326] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:06:19 2022] PUT /identity/v3/projects/82ed179e84e84e219b9581e87f6da3fc/users/1d9fe50ddcbc4ed1bfec2d087be521a7/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:19.568473 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6516aeb-f6f8-41ff-be2f-be4c9e119fde None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:19.568757 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6516aeb-f6f8-41ff-be2f-be4c9e119fde None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:19.569068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6516aeb-f6f8-41ff-be2f-be4c9e119fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:19.601968 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e6516aeb-f6f8-41ff-be2f-be4c9e119fde None None] MFA Rules not processed for user `1d9fe50ddcbc4ed1bfec2d087be521a7`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:19.603360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e6516aeb-f6f8-41ff-be2f-be4c9e119fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:19.674138 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1148/2327] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:19.679302 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98368e31-b556-4f21-be5d-c5506d7f3562 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:19.744245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98368e31-b556-4f21-be5d-c5506d7f3562 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:19.749062 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b508c-191b-4ba0-82bd-f5879170356e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:19.749668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b508c-191b-4ba0-82bd-f5879170356e admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:19.749942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b508c-191b-4ba0-82bd-f5879170356e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:19.750211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b508c-191b-4ba0-82bd-f5879170356e admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:19.750835 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b508c-191b-4ba0-82bd-f5879170356e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:19.752894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b508c-191b-4ba0-82bd-f5879170356e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:19.882986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1180/2328] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:06:19 2022] POST /identity/v3/projects => generated 381 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:06:19.887345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6516aeb-f6f8-41ff-be2f-be4c9e119fde None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:19.898975 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6516aeb-f6f8-41ff-be2f-be4c9e119fde None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:19.960051 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea9abc-7d0a-41c1-8232-b6fe685352b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:19.960719 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea9abc-7d0a-41c1-8232-b6fe685352b9 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:19.961107 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea9abc-7d0a-41c1-8232-b6fe685352b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:19.961389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea9abc-7d0a-41c1-8232-b6fe685352b9 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:19.962197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dea9abc-7d0a-41c1-8232-b6fe685352b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:19.964152 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dea9abc-7d0a-41c1-8232-b6fe685352b9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:20.133582 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1149/2329] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:06:19 2022] POST /identity/v3/users => generated 297 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:06:20.138657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b508c-191b-4ba0-82bd-f5879170356e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:20.150781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b508c-191b-4ba0-82bd-f5879170356e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:20.155480 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2590e2-199c-4a4c-9353-f2982b6ff05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:20.156093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2590e2-199c-4a4c-9353-f2982b6ff05d admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:20.156394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2590e2-199c-4a4c-9353-f2982b6ff05d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:20.156666 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2590e2-199c-4a4c-9353-f2982b6ff05d admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:20.157309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2590e2-199c-4a4c-9353-f2982b6ff05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:20.159064 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2590e2-199c-4a4c-9353-f2982b6ff05d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:20.170567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1181/2330] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:06:20.177981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea9abc-7d0a-41c1-8232-b6fe685352b9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:20.188808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea9abc-7d0a-41c1-8232-b6fe685352b9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:20.193683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade818b0-762c-410e-af3f-0fab753999ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:20.194364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade818b0-762c-410e-af3f-0fab753999ca admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:20.194638 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade818b0-762c-410e-af3f-0fab753999ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:20.194947 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade818b0-762c-410e-af3f-0fab753999ca admin admin] PATH_INFO: `/v3/projects/5a7982139c674520a62c8d89663f9a89/users/4e138b302eee4dc9ad2fe6261b94f417/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:20.215318 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade818b0-762c-410e-af3f-0fab753999ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a7982139c674520a62c8d89663f9a89, user_id=4e138b302eee4dc9ad2fe6261b94f417, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:20.218155 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade818b0-762c-410e-af3f-0fab753999ca admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:20.220703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:20.359082 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1150/2331] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:06:20 2022] PUT /identity/v3/projects/5a7982139c674520a62c8d89663f9a89/users/4e138b302eee4dc9ad2fe6261b94f417/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:20.363163 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2590e2-199c-4a4c-9353-f2982b6ff05d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:20.375844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2590e2-199c-4a4c-9353-f2982b6ff05d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:20.446352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1392bbd-7fff-4b6b-9d74-5d3648bc53be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:20.447020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1392bbd-7fff-4b6b-9d74-5d3648bc53be admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:20.447344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1392bbd-7fff-4b6b-9d74-5d3648bc53be admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:20.447621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1392bbd-7fff-4b6b-9d74-5d3648bc53be admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:20.448271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1392bbd-7fff-4b6b-9d74-5d3648bc53be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:20.450032 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1392bbd-7fff-4b6b-9d74-5d3648bc53be admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:20.462641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1182/2332] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:20.467247 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade818b0-762c-410e-af3f-0fab753999ca admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:20.478970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade818b0-762c-410e-af3f-0fab753999ca admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:20.486086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edfe85-5eef-414a-a26e-b9cfa50d2bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:20.486761 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edfe85-5eef-414a-a26e-b9cfa50d2bc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:20.487068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edfe85-5eef-414a-a26e-b9cfa50d2bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:20.487374 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edfe85-5eef-414a-a26e-b9cfa50d2bc7 admin admin] PATH_INFO: `/v3/projects/5a7982139c674520a62c8d89663f9a89/users/4e138b302eee4dc9ad2fe6261b94f417/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:20.492178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edfe85-5eef-414a-a26e-b9cfa50d2bc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a7982139c674520a62c8d89663f9a89, user_id=4e138b302eee4dc9ad2fe6261b94f417, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:20.495306 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edfe85-5eef-414a-a26e-b9cfa50d2bc7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:20.508067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1151/2333] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:06:20 2022] PUT /identity/v3/projects/5a7982139c674520a62c8d89663f9a89/users/4e138b302eee4dc9ad2fe6261b94f417/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:06:20.523814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ccf1d-ff85-4442-947d-bbe031248785 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:20.524077 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ccf1d-ff85-4442-947d-bbe031248785 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:20.524346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ccf1d-ff85-4442-947d-bbe031248785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:20.566355 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c63ccf1d-ff85-4442-947d-bbe031248785 None None] MFA Rules not processed for user `4e138b302eee4dc9ad2fe6261b94f417`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:20.567712 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c63ccf1d-ff85-4442-947d-bbe031248785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:20.638492 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:20.650429 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1183/2334] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:20.666240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edfe85-5eef-414a-a26e-b9cfa50d2bc7 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:20.670916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-88edfe85-5eef-414a-a26e-b9cfa50d2bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:20.766726 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edfe85-5eef-414a-a26e-b9cfa50d2bc7 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:20.771207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f2c49-f61b-4ec0-9f68-1ee5518cd35c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:20.782664 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f2c49-f61b-4ec0-9f68-1ee5518cd35c service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:20.782664 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f2c49-f61b-4ec0-9f68-1ee5518cd35c service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:20.782664 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f2c49-f61b-4ec0-9f68-1ee5518cd35c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:20.784893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f2c49-f61b-4ec0-9f68-1ee5518cd35c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:20.786870 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f2c49-f61b-4ec0-9f68-1ee5518cd35c service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:20.799754 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1152/2335] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:20 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:21.062981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ccf1d-ff85-4442-947d-bbe031248785 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:21.073392 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ccf1d-ff85-4442-947d-bbe031248785 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:21.077203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a3755-8960-4ef8-800b-121bc6bb3080 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:21.077800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a3755-8960-4ef8-800b-121bc6bb3080 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:21.078118 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a3755-8960-4ef8-800b-121bc6bb3080 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:21.078385 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a3755-8960-4ef8-800b-121bc6bb3080 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:21.135208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a3755-8960-4ef8-800b-121bc6bb3080 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:21.136630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a3755-8960-4ef8-800b-121bc6bb3080 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:21.145565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1184/2336] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:22.007119 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f2c49-f61b-4ec0-9f68-1ee5518cd35c service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:22.077004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f2c49-f61b-4ec0-9f68-1ee5518cd35c service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:22.082494 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7053f-8be6-442d-a9a6-1d3e3e6c5ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:22.083342 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7053f-8be6-442d-a9a6-1d3e3e6c5ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:22.083649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7053f-8be6-442d-a9a6-1d3e3e6c5ded admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:22.083915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7053f-8be6-442d-a9a6-1d3e3e6c5ded admin admin] PATH_INFO: `/v3/users/1d9fe50ddcbc4ed1bfec2d087be521a7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:22.085260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7053f-8be6-442d-a9a6-1d3e3e6c5ded admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d9fe50ddcbc4ed1bfec2d087be521a7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:22.087358 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7053f-8be6-442d-a9a6-1d3e3e6c5ded admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:22.354661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e3a7053f-8be6-442d-a9a6-1d3e3e6c5ded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d9fe50ddcbc4ed1bfec2d087be521a7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:22.540890 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e3a7053f-8be6-442d-a9a6-1d3e3e6c5ded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d9fe50ddcbc4ed1bfec2d087be521a7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:22.560997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e3a7053f-8be6-442d-a9a6-1d3e3e6c5ded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d9fe50ddcbc4ed1bfec2d087be521a7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:22.561776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e3a7053f-8be6-442d-a9a6-1d3e3e6c5ded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d9fe50ddcbc4ed1bfec2d087be521a7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:22.568951 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e3a7053f-8be6-442d-a9a6-1d3e3e6c5ded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d9fe50ddcbc4ed1bfec2d087be521a7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:22.575612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1153/2337] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:06:22 2022] DELETE /identity/v3/users/1d9fe50ddcbc4ed1bfec2d087be521a7 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:22.580677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2385c3-7d21-4fc3-932b-f8b317d8f05d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:22.581188 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2385c3-7d21-4fc3-932b-f8b317d8f05d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:22.581401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2385c3-7d21-4fc3-932b-f8b317d8f05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:22.602240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ca2385c3-7d21-4fc3-932b-f8b317d8f05d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:22.603815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ca2385c3-7d21-4fc3-932b-f8b317d8f05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:22.664182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1185/2338] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:22.674743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7053f-8be6-442d-a9a6-1d3e3e6c5ded admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:22.676225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e3a7053f-8be6-442d-a9a6-1d3e3e6c5ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:22.681527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:22.736287 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7053f-8be6-442d-a9a6-1d3e3e6c5ded admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:22.739929 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cb41f-32a5-4736-a7b8-7b2b44a87d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:22.740544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cb41f-32a5-4736-a7b8-7b2b44a87d82 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:22.740820 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cb41f-32a5-4736-a7b8-7b2b44a87d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:22.741090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cb41f-32a5-4736-a7b8-7b2b44a87d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:22.751001 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cb41f-32a5-4736-a7b8-7b2b44a87d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:22.756715 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cb41f-32a5-4736-a7b8-7b2b44a87d82 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:22.764238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1154/2339] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:23.343294 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2385c3-7d21-4fc3-932b-f8b317d8f05d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:23.345017 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ca2385c3-7d21-4fc3-932b-f8b317d8f05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:23.362588 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2385c3-7d21-4fc3-932b-f8b317d8f05d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:23.369960 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb693a-e603-4595-9385-69a1a9374304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:23.370678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb693a-e603-4595-9385-69a1a9374304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:23.371007 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb693a-e603-4595-9385-69a1a9374304 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:23.371286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb693a-e603-4595-9385-69a1a9374304 admin admin] PATH_INFO: `/v3/projects/82ed179e84e84e219b9581e87f6da3fc` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:23.374156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb693a-e603-4595-9385-69a1a9374304 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ed179e84e84e219b9581e87f6da3fc)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:23.376543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb693a-e603-4595-9385-69a1a9374304 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:23.502297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-83fb693a-e603-4595-9385-69a1a9374304 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ed179e84e84e219b9581e87f6da3fc', 'request_id': 'req-83fb693a-e603-4595-9385-69a1a9374304'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:23.607388 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-83fb693a-e603-4595-9385-69a1a9374304 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ed179e84e84e219b9581e87f6da3fc', 'request_id': 'req-83fb693a-e603-4595-9385-69a1a9374304'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:23.617032 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-83fb693a-e603-4595-9385-69a1a9374304 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ed179e84e84e219b9581e87f6da3fc', 'request_id': 'req-83fb693a-e603-4595-9385-69a1a9374304'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:23.633355 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-83fb693a-e603-4595-9385-69a1a9374304 admin admin] The token cache is being invalidate because project 82ed179e84e84e219b9581e87f6da3fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:06:23.633876 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-83fb693a-e603-4595-9385-69a1a9374304 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:23.634940 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-83fb693a-e603-4595-9385-69a1a9374304 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:23.637499 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1186/2340] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:06:23 2022] DELETE /identity/v3/projects/82ed179e84e84e219b9581e87f6da3fc => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:23.641671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cb41f-32a5-4736-a7b8-7b2b44a87d82 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:23.653289 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cb41f-32a5-4736-a7b8-7b2b44a87d82 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:23.711639 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bde8ef-3a92-4a59-b0a0-c8c640746685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:23.715379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bde8ef-3a92-4a59-b0a0-c8c640746685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:23.715379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bde8ef-3a92-4a59-b0a0-c8c640746685 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:23.715379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bde8ef-3a92-4a59-b0a0-c8c640746685 admin admin] PATH_INFO: `/v3/users/4e138b302eee4dc9ad2fe6261b94f417` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:23.715379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bde8ef-3a92-4a59-b0a0-c8c640746685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e138b302eee4dc9ad2fe6261b94f417)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:23.715904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bde8ef-3a92-4a59-b0a0-c8c640746685 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:24.009177 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d9bde8ef-3a92-4a59-b0a0-c8c640746685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e138b302eee4dc9ad2fe6261b94f417'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:24.101412 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d9bde8ef-3a92-4a59-b0a0-c8c640746685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e138b302eee4dc9ad2fe6261b94f417'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:24.115329 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d9bde8ef-3a92-4a59-b0a0-c8c640746685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e138b302eee4dc9ad2fe6261b94f417'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:24.116106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d9bde8ef-3a92-4a59-b0a0-c8c640746685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e138b302eee4dc9ad2fe6261b94f417'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:24.124965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d9bde8ef-3a92-4a59-b0a0-c8c640746685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e138b302eee4dc9ad2fe6261b94f417'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:24.131544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1155/2341] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:06:23 2022] DELETE /identity/v3/users/4e138b302eee4dc9ad2fe6261b94f417 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:24.694115 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb693a-e603-4595-9385-69a1a9374304 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:24.696303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-83fb693a-e603-4595-9385-69a1a9374304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:24.711756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb693a-e603-4595-9385-69a1a9374304 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:24.731840 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:24.777253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a188820-ad78-4774-8e58-2bf690947c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:24.777898 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a188820-ad78-4774-8e58-2bf690947c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:24.778184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a188820-ad78-4774-8e58-2bf690947c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:24.778644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a188820-ad78-4774-8e58-2bf690947c6d admin admin] PATH_INFO: `/v3/projects/5a7982139c674520a62c8d89663f9a89` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:24.780503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a188820-ad78-4774-8e58-2bf690947c6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a7982139c674520a62c8d89663f9a89)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:24.782538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a188820-ad78-4774-8e58-2bf690947c6d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:24.925601 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6a188820-ad78-4774-8e58-2bf690947c6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a7982139c674520a62c8d89663f9a89', 'request_id': 'req-6a188820-ad78-4774-8e58-2bf690947c6d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:25.035355 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6a188820-ad78-4774-8e58-2bf690947c6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a7982139c674520a62c8d89663f9a89', 'request_id': 'req-6a188820-ad78-4774-8e58-2bf690947c6d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:25.042067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6a188820-ad78-4774-8e58-2bf690947c6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a7982139c674520a62c8d89663f9a89', 'request_id': 'req-6a188820-ad78-4774-8e58-2bf690947c6d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:25.052355 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6a188820-ad78-4774-8e58-2bf690947c6d admin admin] The token cache is being invalidate because project 5a7982139c674520a62c8d89663f9a89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:06:25.052689 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6a188820-ad78-4774-8e58-2bf690947c6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:25.053339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6a188820-ad78-4774-8e58-2bf690947c6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:25.054828 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1187/2342] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:06:24 2022] DELETE /identity/v3/projects/5a7982139c674520a62c8d89663f9a89 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:25.059650 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d6515-6021-4ff2-9fc5-8740ba7381ab None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:25.060116 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d6515-6021-4ff2-9fc5-8740ba7381ab None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:25.060311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d6515-6021-4ff2-9fc5-8740ba7381ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:25.080978 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-3d9d6515-6021-4ff2-9fc5-8740ba7381ab None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:25.082171 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3d9d6515-6021-4ff2-9fc5-8740ba7381ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:25.104547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:25.147733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1156/2343] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:06:25.156365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23ba52-24aa-4bac-8e7b-d351705a5a58 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:25.156657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23ba52-24aa-4bac-8e7b-d351705a5a58 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:25.156943 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23ba52-24aa-4bac-8e7b-d351705a5a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:25.176914 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ab23ba52-24aa-4bac-8e7b-d351705a5a58 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:25.178237 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ab23ba52-24aa-4bac-8e7b-d351705a5a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:25.186313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1188/2344] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:25.190787 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d6515-6021-4ff2-9fc5-8740ba7381ab None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:25.201669 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d6515-6021-4ff2-9fc5-8740ba7381ab None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:25.206563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62477142-0809-4900-be3f-50f672113576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:25.207217 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62477142-0809-4900-be3f-50f672113576 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:25.207895 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62477142-0809-4900-be3f-50f672113576 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:25.207895 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62477142-0809-4900-be3f-50f672113576 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:25.208591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62477142-0809-4900-be3f-50f672113576 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:06:25.208834 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62477142-0809-4900-be3f-50f672113576 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:25.211069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62477142-0809-4900-be3f-50f672113576 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:25.224494 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1157/2345] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:06:25.228859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23ba52-24aa-4bac-8e7b-d351705a5a58 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:25.238636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23ba52-24aa-4bac-8e7b-d351705a5a58 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:25.243524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f60c97-205b-4fba-8de7-25ab7d018340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:25.244156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f60c97-205b-4fba-8de7-25ab7d018340 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:25.244429 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f60c97-205b-4fba-8de7-25ab7d018340 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:25.244697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f60c97-205b-4fba-8de7-25ab7d018340 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:25.245357 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f60c97-205b-4fba-8de7-25ab7d018340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:25.247304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f60c97-205b-4fba-8de7-25ab7d018340 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:25.364814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1189/2346] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:06:25 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:06:25.370173 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62477142-0809-4900-be3f-50f672113576 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:25.381052 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62477142-0809-4900-be3f-50f672113576 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:25.439911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c85487-d533-4a6a-a243-0d2525d64c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:25.440529 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c85487-d533-4a6a-a243-0d2525d64c1b admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:25.440805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c85487-d533-4a6a-a243-0d2525d64c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:25.441108 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c85487-d533-4a6a-a243-0d2525d64c1b admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:25.441819 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c85487-d533-4a6a-a243-0d2525d64c1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:25.443836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c85487-d533-4a6a-a243-0d2525d64c1b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:25.557479 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1158/2347] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:06:25 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:06:25.560349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f60c97-205b-4fba-8de7-25ab7d018340 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:25.571640 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f60c97-205b-4fba-8de7-25ab7d018340 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:25.577081 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3684c-dc56-4749-8793-b0d16b6f1bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:25.577688 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3684c-dc56-4749-8793-b0d16b6f1bfa admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:25.579339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3684c-dc56-4749-8793-b0d16b6f1bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:25.579339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3684c-dc56-4749-8793-b0d16b6f1bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:25.579339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3684c-dc56-4749-8793-b0d16b6f1bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:25.580523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3684c-dc56-4749-8793-b0d16b6f1bfa admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:25.590637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1190/2348] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:25.594804 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c85487-d533-4a6a-a243-0d2525d64c1b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:25.604749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c85487-d533-4a6a-a243-0d2525d64c1b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:25.610660 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4351961-4072-41d9-87b7-d351a43cda9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:25.610660 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4351961-4072-41d9-87b7-d351a43cda9e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:25.610660 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4351961-4072-41d9-87b7-d351a43cda9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:25.610660 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4351961-4072-41d9-87b7-d351a43cda9e admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:25.611382 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4351961-4072-41d9-87b7-d351a43cda9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:25.612917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4351961-4072-41d9-87b7-d351a43cda9e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:25.627772 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1159/2349] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:25.637677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3684c-dc56-4749-8793-b0d16b6f1bfa admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:25.649876 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3684c-dc56-4749-8793-b0d16b6f1bfa admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:25.654693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af737aac-afba-428c-b81b-4d8f615e918d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:25.655615 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af737aac-afba-428c-b81b-4d8f615e918d admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:25.655753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af737aac-afba-428c-b81b-4d8f615e918d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:25.656010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af737aac-afba-428c-b81b-4d8f615e918d admin admin] PATH_INFO: `/v3/projects/f3145c407b92488083921e8e085d137a/users/e559ee76545946e5af1f81d28fc645d1/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:25.672069 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af737aac-afba-428c-b81b-4d8f615e918d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3145c407b92488083921e8e085d137a, user_id=e559ee76545946e5af1f81d28fc645d1, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:25.690941 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af737aac-afba-428c-b81b-4d8f615e918d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:25.790431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1191/2350] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:06:25 2022] PUT /identity/v3/projects/f3145c407b92488083921e8e085d137a/users/e559ee76545946e5af1f81d28fc645d1/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:25.801422 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c31c6-611b-470f-ac44-f1d9ea16f42c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:25.801865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c31c6-611b-470f-ac44-f1d9ea16f42c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:25.802146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c31c6-611b-470f-ac44-f1d9ea16f42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:25.833948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8a3c31c6-611b-470f-ac44-f1d9ea16f42c None None] MFA Rules not processed for user `e559ee76545946e5af1f81d28fc645d1`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:25.835197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8a3c31c6-611b-470f-ac44-f1d9ea16f42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:25.940547 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1160/2351] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:06:25.955394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af737aac-afba-428c-b81b-4d8f615e918d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:25.956031 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-af737aac-afba-428c-b81b-4d8f615e918d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:26.018110 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:26.032795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af737aac-afba-428c-b81b-4d8f615e918d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:26.037719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f0617-051a-4274-8756-98069b229f7c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b060c70e1b440186eeac37d9c34567', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:26.038349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f0617-051a-4274-8756-98069b229f7c service glance] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:26.038628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f0617-051a-4274-8756-98069b229f7c service glance] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:26.038920 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f0617-051a-4274-8756-98069b229f7c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:26.049263 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f0617-051a-4274-8756-98069b229f7c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:26.050750 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f0617-051a-4274-8756-98069b229f7c service glance] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:26.060890 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1192/2352] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:35.936403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:36.209145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=6 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:40.256763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c31c6-611b-470f-ac44-f1d9ea16f42c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:40.258327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8a3c31c6-611b-470f-ac44-f1d9ea16f42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:40.315744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c31c6-611b-470f-ac44-f1d9ea16f42c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:40.319484 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541022b-5b17-4db7-b52c-bae8792643e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:40.320116 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541022b-5b17-4db7-b52c-bae8792643e9 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:40.320390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541022b-5b17-4db7-b52c-bae8792643e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:40.320654 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541022b-5b17-4db7-b52c-bae8792643e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:40.322341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1541022b-5b17-4db7-b52c-bae8792643e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:40.387027 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541022b-5b17-4db7-b52c-bae8792643e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:40.388457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541022b-5b17-4db7-b52c-bae8792643e9 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:40.397976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1161/2353] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:42.599142 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f0617-051a-4274-8756-98069b229f7c service glance] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:42.608680 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f0617-051a-4274-8756-98069b229f7c service glance] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:42.613984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc322b0-7602-4468-904c-fe419fa291cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:42.614781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc322b0-7602-4468-904c-fe419fa291cb service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:42.615315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc322b0-7602-4468-904c-fe419fa291cb service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:42.615786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc322b0-7602-4468-904c-fe419fa291cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:42.617106 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-bbc322b0-7602-4468-904c-fe419fa291cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:42.692346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc322b0-7602-4468-904c-fe419fa291cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:42.693944 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc322b0-7602-4468-904c-fe419fa291cb service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:42.705445 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1193/2354] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:44.705214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ec88a-8594-4781-b60f-45ec5a05b91b None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:44.705749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ec88a-8594-4781-b60f-45ec5a05b91b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:44.705963 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ec88a-8594-4781-b60f-45ec5a05b91b None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:44.707180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1162/2355] 10.222.0.91 () {54 vars in 921 bytes} [Thu Sep 8 14:06:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:06:44.714525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0a31c-eea8-4e93-8724-6b8185652160 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:44.715093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0a31c-eea8-4e93-8724-6b8185652160 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:44.715093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0a31c-eea8-4e93-8724-6b8185652160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:44.764481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d6c0a31c-eea8-4e93-8724-6b8185652160 None None] MFA Rules not processed for user `f9506139d5744fe69153e7e30a05b436`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:44.767499 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d6c0a31c-eea8-4e93-8724-6b8185652160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:44.876639 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ec88a-8594-4781-b60f-45ec5a05b91b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:44.877693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1194/2356] 10.222.0.91 () {58 vars in 1018 bytes} [Thu Sep 8 14:06:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:44.881268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a54ec88a-8594-4781-b60f-45ec5a05b91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:44.919455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ec88a-8594-4781-b60f-45ec5a05b91b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:44.926367 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0a31c-eea8-4e93-8724-6b8185652160 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:44.928691 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d6c0a31c-eea8-4e93-8724-6b8185652160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:44.929428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e558a16-e277-4cc1-b26c-8bf96a771561 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9506139d5744fe69153e7e30a05b436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:44.930242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e558a16-e277-4cc1-b26c-8bf96a771561 service placement] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:44.932655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e558a16-e277-4cc1-b26c-8bf96a771561 service placement] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:44.933147 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e558a16-e277-4cc1-b26c-8bf96a771561 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:44.943050 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e558a16-e277-4cc1-b26c-8bf96a771561 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:44.945560 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e558a16-e277-4cc1-b26c-8bf96a771561 service placement] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:44.946553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0a31c-eea8-4e93-8724-6b8185652160 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:44.953266 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff95e07-47f4-4771-845a-6b2dd9d28f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:44.954397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff95e07-47f4-4771-845a-6b2dd9d28f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:44.954995 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff95e07-47f4-4771-845a-6b2dd9d28f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:44.955445 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff95e07-47f4-4771-845a-6b2dd9d28f4e admin admin] PATH_INFO: `/v3/users/7b403bdc9cad4783b72bbf174199363c` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:44.957259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff95e07-47f4-4771-845a-6b2dd9d28f4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b403bdc9cad4783b72bbf174199363c)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:44.959908 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff95e07-47f4-4771-845a-6b2dd9d28f4e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:44.966579 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1163/2357] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:45.210946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-dff95e07-47f4-4771-845a-6b2dd9d28f4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b403bdc9cad4783b72bbf174199363c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:45.286713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-dff95e07-47f4-4771-845a-6b2dd9d28f4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b403bdc9cad4783b72bbf174199363c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:45.287463 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-dff95e07-47f4-4771-845a-6b2dd9d28f4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b403bdc9cad4783b72bbf174199363c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:45.295376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-dff95e07-47f4-4771-845a-6b2dd9d28f4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b403bdc9cad4783b72bbf174199363c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:45.310339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-dff95e07-47f4-4771-845a-6b2dd9d28f4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b403bdc9cad4783b72bbf174199363c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:45.311013 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:45.317653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1195/2358] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:06:44 2022] DELETE /identity/v3/users/7b403bdc9cad4783b72bbf174199363c => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:45.659544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e558a16-e277-4cc1-b26c-8bf96a771561 service placement] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:45.672916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e558a16-e277-4cc1-b26c-8bf96a771561 service placement] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:45.751600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1fb5e-1119-45c4-bdd4-0c2c4133e1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:45.752659 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff95e07-47f4-4771-845a-6b2dd9d28f4e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:45.753431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-dff95e07-47f4-4771-845a-6b2dd9d28f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:45.754334 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1fb5e-1119-45c4-bdd4-0c2c4133e1dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:45.755142 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1fb5e-1119-45c4-bdd4-0c2c4133e1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:45.755142 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1fb5e-1119-45c4-bdd4-0c2c4133e1dd admin admin] PATH_INFO: `/v3/projects/4d0ed4e4f54c47369721e871f0dec518` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:45.756409 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1fb5e-1119-45c4-bdd4-0c2c4133e1dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d0ed4e4f54c47369721e871f0dec518)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:45.758464 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1fb5e-1119-45c4-bdd4-0c2c4133e1dd admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:45.829155 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff95e07-47f4-4771-845a-6b2dd9d28f4e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:45.834095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcb177-f04b-4607-96e6-dd23a7f06b82 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b060c70e1b440186eeac37d9c34567', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:45.834738 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcb177-f04b-4607-96e6-dd23a7f06b82 service glance] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:45.835060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcb177-f04b-4607-96e6-dd23a7f06b82 service glance] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:45.838640 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcb177-f04b-4607-96e6-dd23a7f06b82 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:45.838640 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-92bcb177-f04b-4607-96e6-dd23a7f06b82 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:45.873058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:45.887148 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-06a1fb5e-1119-45c4-bdd4-0c2c4133e1dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d0ed4e4f54c47369721e871f0dec518', 'request_id': 'req-06a1fb5e-1119-45c4-bdd4-0c2c4133e1dd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:45.896505 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-06a1fb5e-1119-45c4-bdd4-0c2c4133e1dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d0ed4e4f54c47369721e871f0dec518', 'request_id': 'req-06a1fb5e-1119-45c4-bdd4-0c2c4133e1dd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:45.908222 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-06a1fb5e-1119-45c4-bdd4-0c2c4133e1dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d0ed4e4f54c47369721e871f0dec518', 'request_id': 'req-06a1fb5e-1119-45c4-bdd4-0c2c4133e1dd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:45.918756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcb177-f04b-4607-96e6-dd23a7f06b82 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:45.920330 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcb177-f04b-4607-96e6-dd23a7f06b82 service glance] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:45.930525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1196/2359] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:45.931963 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcb177-f04b-4607-96e6-dd23a7f06b82 service glance] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:45.942451 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-92bcb177-f04b-4607-96e6-dd23a7f06b82 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:45.960631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcb177-f04b-4607-96e6-dd23a7f06b82 service glance] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:46.012263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-06a1fb5e-1119-45c4-bdd4-0c2c4133e1dd admin admin] The token cache is being invalidate because project 4d0ed4e4f54c47369721e871f0dec518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:06:46.012878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-06a1fb5e-1119-45c4-bdd4-0c2c4133e1dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:46.014269 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-06a1fb5e-1119-45c4-bdd4-0c2c4133e1dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:46.022227 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1164/2360] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:06:45 2022] DELETE /identity/v3/projects/4d0ed4e4f54c47369721e871f0dec518 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:46.024650 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adce6e7-cf7f-493b-8457-c1cf8a8ad46f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:46.025158 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adce6e7-cf7f-493b-8457-c1cf8a8ad46f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:46.025608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adce6e7-cf7f-493b-8457-c1cf8a8ad46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:46.052024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918285c7-c3d1-4e18-bd51-1266ec25d72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:46.052982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918285c7-c3d1-4e18-bd51-1266ec25d72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:46.053266 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918285c7-c3d1-4e18-bd51-1266ec25d72b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:46.053543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918285c7-c3d1-4e18-bd51-1266ec25d72b admin admin] PATH_INFO: `/v3/users/e559ee76545946e5af1f81d28fc645d1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:46.054725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8adce6e7-cf7f-493b-8457-c1cf8a8ad46f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:46.054846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918285c7-c3d1-4e18-bd51-1266ec25d72b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e559ee76545946e5af1f81d28fc645d1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:46.056379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8adce6e7-cf7f-493b-8457-c1cf8a8ad46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:46.057858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918285c7-c3d1-4e18-bd51-1266ec25d72b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:46.078939 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1165/2361] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:06:46.082939 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1e1c3-969f-4ce8-9f1a-7f56fbe6d6e3 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:46.083224 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1e1c3-969f-4ce8-9f1a-7f56fbe6d6e3 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:46.083499 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1e1c3-969f-4ce8-9f1a-7f56fbe6d6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:46.117364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-59c1e1c3-969f-4ce8-9f1a-7f56fbe6d6e3 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:46.120161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-59c1e1c3-969f-4ce8-9f1a-7f56fbe6d6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:46.134472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1166/2362] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:46.135618 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1e1c3-969f-4ce8-9f1a-7f56fbe6d6e3 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:46.148310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1e1c3-969f-4ce8-9f1a-7f56fbe6d6e3 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:46.154050 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431fb04-5b44-4f87-b25a-843ff454c1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:46.155250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431fb04-5b44-4f87-b25a-843ff454c1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:46.155628 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431fb04-5b44-4f87-b25a-843ff454c1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:46.156048 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431fb04-5b44-4f87-b25a-843ff454c1e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:46.156830 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431fb04-5b44-4f87-b25a-843ff454c1e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:06:46.157254 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431fb04-5b44-4f87-b25a-843ff454c1e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:46.159230 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431fb04-5b44-4f87-b25a-843ff454c1e3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:46.181262 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1167/2363] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:06:46.182364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431fb04-5b44-4f87-b25a-843ff454c1e3 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:46.193998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431fb04-5b44-4f87-b25a-843ff454c1e3 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:46.199992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a694d6-af21-485b-8fef-b657bd66fb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:46.200868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a694d6-af21-485b-8fef-b657bd66fb19 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:46.201336 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a694d6-af21-485b-8fef-b657bd66fb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:46.201768 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a694d6-af21-485b-8fef-b657bd66fb19 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:46.202715 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a694d6-af21-485b-8fef-b657bd66fb19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:46.205087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a694d6-af21-485b-8fef-b657bd66fb19 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:46.320418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1168/2364] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:06:46 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:06:46.325636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a694d6-af21-485b-8fef-b657bd66fb19 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:46.329157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-918285c7-c3d1-4e18-bd51-1266ec25d72b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e559ee76545946e5af1f81d28fc645d1'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:46.337674 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:46.412336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-918285c7-c3d1-4e18-bd51-1266ec25d72b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e559ee76545946e5af1f81d28fc645d1'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:46.413016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-918285c7-c3d1-4e18-bd51-1266ec25d72b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e559ee76545946e5af1f81d28fc645d1'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:46.428160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-918285c7-c3d1-4e18-bd51-1266ec25d72b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e559ee76545946e5af1f81d28fc645d1'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:46.460766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-918285c7-c3d1-4e18-bd51-1266ec25d72b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e559ee76545946e5af1f81d28fc645d1'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:46.469722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1197/2365] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:06:45 2022] DELETE /identity/v3/users/e559ee76545946e5af1f81d28fc645d1 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:46.474348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6e03f-f91a-4259-8f39-213adac03367 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:46.481678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a694d6-af21-485b-8fef-b657bd66fb19 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:46.491510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6e03f-f91a-4259-8f39-213adac03367 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:46.492159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6e03f-f91a-4259-8f39-213adac03367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:46.549476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-acf6e03f-f91a-4259-8f39-213adac03367 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:46.559644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-acf6e03f-f91a-4259-8f39-213adac03367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:46.613576 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cbabe-4742-4e1f-bebe-710d15219082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:46.618013 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cbabe-4742-4e1f-bebe-710d15219082 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:46.618013 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cbabe-4742-4e1f-bebe-710d15219082 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:46.618013 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cbabe-4742-4e1f-bebe-710d15219082 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:46.618013 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cbabe-4742-4e1f-bebe-710d15219082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:46.619367 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cbabe-4742-4e1f-bebe-710d15219082 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:46.658948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1198/2366] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:06:46.660351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9d830-31f8-4367-9b13-7b09f99679e6 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:46.660699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9d830-31f8-4367-9b13-7b09f99679e6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:46.661051 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9d830-31f8-4367-9b13-7b09f99679e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:46.685287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3fd9d830-31f8-4367-9b13-7b09f99679e6 None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:46.686912 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3fd9d830-31f8-4367-9b13-7b09f99679e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:46.738380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1169/2367] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:06:46 2022] POST /identity/v3/users => generated 286 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:06:46.739612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cbabe-4742-4e1f-bebe-710d15219082 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:46.741849 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a85cbabe-4742-4e1f-bebe-710d15219082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:46.756741 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1199/2368] 10.222.0.91 () {58 vars in 1009 bytes} [Thu Sep 8 14:06:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:46.757614 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9d830-31f8-4367-9b13-7b09f99679e6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:46.760028 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3fd9d830-31f8-4367-9b13-7b09f99679e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:46.765946 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:46.787578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9d830-31f8-4367-9b13-7b09f99679e6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:46.793656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bde221-3a8e-4d69-b70a-dfb696bb8a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:46.795315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bde221-3a8e-4d69-b70a-dfb696bb8a1b admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:46.795861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bde221-3a8e-4d69-b70a-dfb696bb8a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:46.796341 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bde221-3a8e-4d69-b70a-dfb696bb8a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:46.797490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bde221-3a8e-4d69-b70a-dfb696bb8a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:46.800152 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bde221-3a8e-4d69-b70a-dfb696bb8a1b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:46.811228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1200/2369] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:46.815261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bde221-3a8e-4d69-b70a-dfb696bb8a1b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:46.817734 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cbabe-4742-4e1f-bebe-710d15219082 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:46.822766 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf43708-a12f-4c31-8c7a-af8b6d51a967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:46.823085 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf43708-a12f-4c31-8c7a-af8b6d51a967 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:46.823422 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf43708-a12f-4c31-8c7a-af8b6d51a967 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:46.823695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf43708-a12f-4c31-8c7a-af8b6d51a967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:46.826144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bde221-3a8e-4d69-b70a-dfb696bb8a1b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:46.830981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b43cf-4013-4e28-8c2e-fb936b7a6904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:46.831565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b43cf-4013-4e28-8c2e-fb936b7a6904 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:46.831840 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b43cf-4013-4e28-8c2e-fb936b7a6904 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:46.832115 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b43cf-4013-4e28-8c2e-fb936b7a6904 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:46.832789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b43cf-4013-4e28-8c2e-fb936b7a6904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:46.834211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf43708-a12f-4c31-8c7a-af8b6d51a967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:46.834380 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b43cf-4013-4e28-8c2e-fb936b7a6904 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:46.835751 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf43708-a12f-4c31-8c7a-af8b6d51a967 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:46.848336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1201/2370] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:46.848540 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1170/2371] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:46.849459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b43cf-4013-4e28-8c2e-fb936b7a6904 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:46.859952 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b43cf-4013-4e28-8c2e-fb936b7a6904 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:46.865275 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade636ac-b48d-4b7a-bd17-f9c4a6d2baf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:46.865979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade636ac-b48d-4b7a-bd17-f9c4a6d2baf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:46.866261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade636ac-b48d-4b7a-bd17-f9c4a6d2baf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:46.866537 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade636ac-b48d-4b7a-bd17-f9c4a6d2baf3 admin admin] PATH_INFO: `/v3/projects/f33dbf40d3e24e1f934b25abe0253ea9/users/29040802b064433f83aee605e0b496c2/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:46.885051 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade636ac-b48d-4b7a-bd17-f9c4a6d2baf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f33dbf40d3e24e1f934b25abe0253ea9, user_id=29040802b064433f83aee605e0b496c2, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:46.887837 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade636ac-b48d-4b7a-bd17-f9c4a6d2baf3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:46.954683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf43708-a12f-4c31-8c7a-af8b6d51a967 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:46.956231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8cf43708-a12f-4c31-8c7a-af8b6d51a967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:46.987648 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1202/2372] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:06:46 2022] PUT /identity/v3/projects/f33dbf40d3e24e1f934b25abe0253ea9/users/29040802b064433f83aee605e0b496c2/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:06:46.998128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fc014-e819-4dea-aa6a-068332b61048 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:46.999152 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fc014-e819-4dea-aa6a-068332b61048 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:47.003466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fc014-e819-4dea-aa6a-068332b61048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:47.039548 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-2a7fc014-e819-4dea-aa6a-068332b61048 None None] MFA Rules not processed for user `29040802b064433f83aee605e0b496c2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:47.043371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2a7fc014-e819-4dea-aa6a-068332b61048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:47.053814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf43708-a12f-4c31-8c7a-af8b6d51a967 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:47.054565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:47.119176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1203/2373] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:06:47.124405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fc014-e819-4dea-aa6a-068332b61048 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:47.148294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f279cc5-1da2-450c-9fec-0c35b069efde service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:47.149512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f279cc5-1da2-450c-9fec-0c35b069efde service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:47.149978 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f279cc5-1da2-450c-9fec-0c35b069efde service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:47.150640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f279cc5-1da2-450c-9fec-0c35b069efde service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:47.232127 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fc014-e819-4dea-aa6a-068332b61048 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:47.249322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cef243-004a-48dc-91f4-d91fab62ffca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:47.250149 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cef243-004a-48dc-91f4-d91fab62ffca admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:47.250500 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cef243-004a-48dc-91f4-d91fab62ffca admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:47.250846 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cef243-004a-48dc-91f4-d91fab62ffca admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:47.261327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cef243-004a-48dc-91f4-d91fab62ffca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:47.261327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cef243-004a-48dc-91f4-d91fab62ffca admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:47.261598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f279cc5-1da2-450c-9fec-0c35b069efde service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:47.261598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f279cc5-1da2-450c-9fec-0c35b069efde service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:47.270039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1171/2374] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 14:06:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 14:06:47.365066 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1204/2375] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:06:47 2022] POST /identity/v3/projects => generated 371 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:06:47.365178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f279cc5-1da2-450c-9fec-0c35b069efde service swift] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:47.365536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cef243-004a-48dc-91f4-d91fab62ffca admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:47.371393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2f279cc5-1da2-450c-9fec-0c35b069efde service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:47.386867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:47.404322 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f279cc5-1da2-450c-9fec-0c35b069efde service swift] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:47.548840 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cef243-004a-48dc-91f4-d91fab62ffca admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:47.553333 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b845c2-cf48-4e81-a09b-b0b98bfbd4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:47.553827 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5795fb1-ef2f-4554-899a-a50dcef9589e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:47.554350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b845c2-cf48-4e81-a09b-b0b98bfbd4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:47.554667 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5795fb1-ef2f-4554-899a-a50dcef9589e admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:47.554979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b845c2-cf48-4e81-a09b-b0b98bfbd4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:47.555090 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5795fb1-ef2f-4554-899a-a50dcef9589e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:47.555173 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b845c2-cf48-4e81-a09b-b0b98bfbd4f9 admin admin] PATH_INFO: `/v3/projects/f3145c407b92488083921e8e085d137a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:47.555261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5795fb1-ef2f-4554-899a-a50dcef9589e admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:47.555960 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5795fb1-ef2f-4554-899a-a50dcef9589e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:47.556834 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b845c2-cf48-4e81-a09b-b0b98bfbd4f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3145c407b92488083921e8e085d137a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:47.557809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5795fb1-ef2f-4554-899a-a50dcef9589e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:47.559088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b845c2-cf48-4e81-a09b-b0b98bfbd4f9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:47.723587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1205/2376] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:06:47 2022] POST /identity/v3/users => generated 292 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:06:47.729111 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5795fb1-ef2f-4554-899a-a50dcef9589e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:47.752663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5795fb1-ef2f-4554-899a-a50dcef9589e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:47.765853 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-90b845c2-cf48-4e81-a09b-b0b98bfbd4f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3145c407b92488083921e8e085d137a', 'request_id': 'req-90b845c2-cf48-4e81-a09b-b0b98bfbd4f9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:47.775844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-90b845c2-cf48-4e81-a09b-b0b98bfbd4f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3145c407b92488083921e8e085d137a', 'request_id': 'req-90b845c2-cf48-4e81-a09b-b0b98bfbd4f9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:47.785254 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-90b845c2-cf48-4e81-a09b-b0b98bfbd4f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3145c407b92488083921e8e085d137a', 'request_id': 'req-90b845c2-cf48-4e81-a09b-b0b98bfbd4f9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:47.810550 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:47.869262 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ebe12-3334-457b-9192-a0cec70d49d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:47.870323 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ebe12-3334-457b-9192-a0cec70d49d0 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:47.870627 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ebe12-3334-457b-9192-a0cec70d49d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:47.870916 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ebe12-3334-457b-9192-a0cec70d49d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:47.871858 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ebe12-3334-457b-9192-a0cec70d49d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:47.874143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ebe12-3334-457b-9192-a0cec70d49d0 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:47.875906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-90b845c2-cf48-4e81-a09b-b0b98bfbd4f9 admin admin] The token cache is being invalidate because project f3145c407b92488083921e8e085d137a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:06:47.876458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-90b845c2-cf48-4e81-a09b-b0b98bfbd4f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:47.877859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-90b845c2-cf48-4e81-a09b-b0b98bfbd4f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:47.882355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1172/2377] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:06:47 2022] DELETE /identity/v3/projects/f3145c407b92488083921e8e085d137a => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:06:47.888976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67393732-c192-4802-9a3f-8e49ed2d01cf None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:47.889631 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67393732-c192-4802-9a3f-8e49ed2d01cf None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:47.889773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1206/2378] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:47.890392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67393732-c192-4802-9a3f-8e49ed2d01cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:47.894020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ebe12-3334-457b-9192-a0cec70d49d0 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:47.897383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b09ebe12-3334-457b-9192-a0cec70d49d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:47.912911 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ebe12-3334-457b-9192-a0cec70d49d0 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:47.922777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4d2ff-2678-4d00-b252-6e2ddef60d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:47.923940 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4d2ff-2678-4d00-b252-6e2ddef60d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:47.924376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4d2ff-2678-4d00-b252-6e2ddef60d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:47.924663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4d2ff-2678-4d00-b252-6e2ddef60d8b admin admin] PATH_INFO: `/v3/projects/a8d43bd71c744641835b6bd91d9d755e/users/07583736988b4d119befe6145f4b8172/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:47.931835 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-67393732-c192-4802-9a3f-8e49ed2d01cf None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:47.933852 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-67393732-c192-4802-9a3f-8e49ed2d01cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:47.944845 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4d2ff-2678-4d00-b252-6e2ddef60d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d43bd71c744641835b6bd91d9d755e, user_id=07583736988b4d119befe6145f4b8172, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:47.945064 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1173/2379] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:47.953259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4d2ff-2678-4d00-b252-6e2ddef60d8b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:47.957010 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5743315-59f0-4a97-aa61-c4d7a6a4f176 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:47.957368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5743315-59f0-4a97-aa61-c4d7a6a4f176 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:47.957721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5743315-59f0-4a97-aa61-c4d7a6a4f176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:47.982837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c5743315-59f0-4a97-aa61-c4d7a6a4f176 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:47.984849 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c5743315-59f0-4a97-aa61-c4d7a6a4f176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:47.996213 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1174/2380] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:48.000240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5743315-59f0-4a97-aa61-c4d7a6a4f176 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:48.010450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5743315-59f0-4a97-aa61-c4d7a6a4f176 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:48.019959 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd43b23-7fa3-424b-b40d-02396adc4dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:48.020592 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd43b23-7fa3-424b-b40d-02396adc4dce admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:48.020867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd43b23-7fa3-424b-b40d-02396adc4dce admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:48.021140 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd43b23-7fa3-424b-b40d-02396adc4dce admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:48.021793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd43b23-7fa3-424b-b40d-02396adc4dce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:06:48.022099 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd43b23-7fa3-424b-b40d-02396adc4dce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:48.024028 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd43b23-7fa3-424b-b40d-02396adc4dce admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:48.043201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1207/2381] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:06:47 2022] PUT /identity/v3/projects/a8d43bd71c744641835b6bd91d9d755e/users/07583736988b4d119befe6145f4b8172/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:48.043602 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1175/2382] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:06:48.044586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4d2ff-2678-4d00-b252-6e2ddef60d8b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:48.063836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd43b23-7fa3-424b-b40d-02396adc4dce admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:48.078935 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd43b23-7fa3-424b-b40d-02396adc4dce admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:48.138097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4d2ff-2678-4d00-b252-6e2ddef60d8b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:48.151224 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e449a06-1beb-4a60-a471-af729f9b5b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:48.153148 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e449a06-1beb-4a60-a471-af729f9b5b46 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:48.156637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e449a06-1beb-4a60-a471-af729f9b5b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:48.156637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e449a06-1beb-4a60-a471-af729f9b5b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:48.156637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e449a06-1beb-4a60-a471-af729f9b5b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:48.156637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e449a06-1beb-4a60-a471-af729f9b5b46 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:48.166477 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27f518-b397-4192-b9dd-ec89afafeb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:48.167201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27f518-b397-4192-b9dd-ec89afafeb08 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:48.167480 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27f518-b397-4192-b9dd-ec89afafeb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:48.167882 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27f518-b397-4192-b9dd-ec89afafeb08 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:48.168518 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27f518-b397-4192-b9dd-ec89afafeb08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:48.169635 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1208/2383] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:48.171646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27f518-b397-4192-b9dd-ec89afafeb08 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:48.173789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e449a06-1beb-4a60-a471-af729f9b5b46 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:48.185170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e449a06-1beb-4a60-a471-af729f9b5b46 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:48.191097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b375f83-7845-4bd8-b209-32f47f3cb879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:48.191832 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b375f83-7845-4bd8-b209-32f47f3cb879 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:48.192113 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b375f83-7845-4bd8-b209-32f47f3cb879 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:48.192393 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b375f83-7845-4bd8-b209-32f47f3cb879 admin admin] PATH_INFO: `/v3/projects/a8d43bd71c744641835b6bd91d9d755e/users/07583736988b4d119befe6145f4b8172/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:48.194503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b375f83-7845-4bd8-b209-32f47f3cb879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d43bd71c744641835b6bd91d9d755e, user_id=07583736988b4d119befe6145f4b8172, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:48.195697 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:48.200153 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b375f83-7845-4bd8-b209-32f47f3cb879 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:48.211146 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1209/2384] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:06:48 2022] PUT /identity/v3/projects/a8d43bd71c744641835b6bd91d9d755e/users/07583736988b4d119befe6145f4b8172/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:06:48.222501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599122a4-6f24-4fbe-836f-a87917f6b164 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:48.225526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599122a4-6f24-4fbe-836f-a87917f6b164 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:48.225526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599122a4-6f24-4fbe-836f-a87917f6b164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:48.259391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-599122a4-6f24-4fbe-836f-a87917f6b164 None None] MFA Rules not processed for user `07583736988b4d119befe6145f4b8172`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:48.260561 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-599122a4-6f24-4fbe-836f-a87917f6b164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:48.303364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1176/2385] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:06:48 2022] POST /identity/v3/projects => generated 375 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:06:48.308087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27f518-b397-4192-b9dd-ec89afafeb08 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:48.359461 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1210/2386] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:48.369172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599122a4-6f24-4fbe-836f-a87917f6b164 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:48.370714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-599122a4-6f24-4fbe-836f-a87917f6b164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:48.389729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:48.401091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27f518-b397-4192-b9dd-ec89afafeb08 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:48.406044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081ed95-64d0-491d-bfcd-41e243c45c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:48.406771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081ed95-64d0-491d-bfcd-41e243c45c5b admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:48.407157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081ed95-64d0-491d-bfcd-41e243c45c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:48.407501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081ed95-64d0-491d-bfcd-41e243c45c5b admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:48.408337 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081ed95-64d0-491d-bfcd-41e243c45c5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:48.410343 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081ed95-64d0-491d-bfcd-41e243c45c5b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:48.450409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599122a4-6f24-4fbe-836f-a87917f6b164 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:48.523262 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b2b9a-5b73-4050-b355-456ed9891fca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:48.524186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b2b9a-5b73-4050-b355-456ed9891fca service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:48.524514 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b2b9a-5b73-4050-b355-456ed9891fca service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:48.524812 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b2b9a-5b73-4050-b355-456ed9891fca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:48.533376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1177/2387] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:06:48 2022] POST /identity/v3/users => generated 288 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:06:48.538460 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081ed95-64d0-491d-bfcd-41e243c45c5b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:48.559323 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081ed95-64d0-491d-bfcd-41e243c45c5b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:48.566996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf0824-01b3-4fc5-b60e-2481a464ad7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:48.567975 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf0824-01b3-4fc5-b60e-2481a464ad7f admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:48.568425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf0824-01b3-4fc5-b60e-2481a464ad7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:48.568885 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf0824-01b3-4fc5-b60e-2481a464ad7f admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:48.570168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf0824-01b3-4fc5-b60e-2481a464ad7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:48.572850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf0824-01b3-4fc5-b60e-2481a464ad7f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:48.586470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1178/2388] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:48.587960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf0824-01b3-4fc5-b60e-2481a464ad7f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:48.599308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf0824-01b3-4fc5-b60e-2481a464ad7f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:48.607220 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816d587-a067-4a63-a47c-e0bacb16eb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:48.608459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816d587-a067-4a63-a47c-e0bacb16eb2a admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:48.609355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816d587-a067-4a63-a47c-e0bacb16eb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:48.610186 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816d587-a067-4a63-a47c-e0bacb16eb2a admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:48.611448 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816d587-a067-4a63-a47c-e0bacb16eb2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:48.611599 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b2b9a-5b73-4050-b355-456ed9891fca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:48.613083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b2b9a-5b73-4050-b355-456ed9891fca service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:48.613802 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816d587-a067-4a63-a47c-e0bacb16eb2a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:48.626672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1179/2389] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:48.627132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1211/2390] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:06:48.631046 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816d587-a067-4a63-a47c-e0bacb16eb2a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:48.643263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816d587-a067-4a63-a47c-e0bacb16eb2a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:48.648869 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972dbea9-1ba6-4581-84d7-82449d17da54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:48.650141 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dbea9-1ba6-4581-84d7-82449d17da54 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:48.650417 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dbea9-1ba6-4581-84d7-82449d17da54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:48.650689 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dbea9-1ba6-4581-84d7-82449d17da54 admin admin] PATH_INFO: `/v3/projects/f88c9d3db3cd472f8c7daaa9a0f27a24/users/ed63a181838342a19137a4f04ff7bf9a/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:48.671460 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972dbea9-1ba6-4581-84d7-82449d17da54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f88c9d3db3cd472f8c7daaa9a0f27a24, user_id=ed63a181838342a19137a4f04ff7bf9a, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:48.674207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972dbea9-1ba6-4581-84d7-82449d17da54 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:48.758538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1180/2391] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:06:48 2022] PUT /identity/v3/projects/f88c9d3db3cd472f8c7daaa9a0f27a24/users/ed63a181838342a19137a4f04ff7bf9a/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:48.779131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69912f-c6c6-4ebf-ae2e-0bec0a8d0f0a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:48.779507 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69912f-c6c6-4ebf-ae2e-0bec0a8d0f0a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:48.779683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69912f-c6c6-4ebf-ae2e-0bec0a8d0f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:48.814004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-9f69912f-c6c6-4ebf-ae2e-0bec0a8d0f0a None None] MFA Rules not processed for user `ed63a181838342a19137a4f04ff7bf9a`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:48.815533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9f69912f-c6c6-4ebf-ae2e-0bec0a8d0f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:48.873438 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972dbea9-1ba6-4581-84d7-82449d17da54 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:48.908928 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1212/2392] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:48.912116 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69912f-c6c6-4ebf-ae2e-0bec0a8d0f0a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:48.932760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:48.955202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972dbea9-1ba6-4581-84d7-82449d17da54 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:48.959100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da29bab-f87c-46a7-9f02-c8dc542aab0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:48.959714 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da29bab-f87c-46a7-9f02-c8dc542aab0a service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:48.960007 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da29bab-f87c-46a7-9f02-c8dc542aab0a service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:48.960260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da29bab-f87c-46a7-9f02-c8dc542aab0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:48.962586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8da29bab-f87c-46a7-9f02-c8dc542aab0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:49.026186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69912f-c6c6-4ebf-ae2e-0bec0a8d0f0a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:49.032847 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:49.033298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede934fe-745e-4ac0-9767-0785de2adc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:49.034046 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede934fe-745e-4ac0-9767-0785de2adc6f admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:49.035335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede934fe-745e-4ac0-9767-0785de2adc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:49.035623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede934fe-745e-4ac0-9767-0785de2adc6f admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:49.036271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede934fe-745e-4ac0-9767-0785de2adc6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:49.038256 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede934fe-745e-4ac0-9767-0785de2adc6f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:49.045264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da29bab-f87c-46a7-9f02-c8dc542aab0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:49.046879 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da29bab-f87c-46a7-9f02-c8dc542aab0a service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:49.057561 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1181/2393] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:06:49.127351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1213/2394] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:06:48 2022] POST /identity/v3/projects => generated 373 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:06:49.133585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da29bab-f87c-46a7-9f02-c8dc542aab0a service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:49.154357 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da29bab-f87c-46a7-9f02-c8dc542aab0a service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:49.231317 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2693c46-1eb7-43bf-b174-ef1ed7601264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:49.232121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2693c46-1eb7-43bf-b174-ef1ed7601264 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:49.232407 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2693c46-1eb7-43bf-b174-ef1ed7601264 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:49.232833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2693c46-1eb7-43bf-b174-ef1ed7601264 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:49.233479 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2693c46-1eb7-43bf-b174-ef1ed7601264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:49.235444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2693c46-1eb7-43bf-b174-ef1ed7601264 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:49.369759 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1182/2395] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:06:49 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:06:49.370872 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede934fe-745e-4ac0-9767-0785de2adc6f admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:49.386137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede934fe-745e-4ac0-9767-0785de2adc6f admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:49.392039 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc97cfb-4993-4f4b-b871-20c2a98c4c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:49.392902 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc97cfb-4993-4f4b-b871-20c2a98c4c0b admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:49.393285 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc97cfb-4993-4f4b-b871-20c2a98c4c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:49.393643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc97cfb-4993-4f4b-b871-20c2a98c4c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:49.394378 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc97cfb-4993-4f4b-b871-20c2a98c4c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:49.397644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc97cfb-4993-4f4b-b871-20c2a98c4c0b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:49.408535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1214/2396] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:49.412705 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2693c46-1eb7-43bf-b174-ef1ed7601264 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:49.424721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2693c46-1eb7-43bf-b174-ef1ed7601264 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:49.430076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a775e80-89e5-425a-a3be-fd5123e1905b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:49.431539 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a775e80-89e5-425a-a3be-fd5123e1905b admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:49.431924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a775e80-89e5-425a-a3be-fd5123e1905b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:49.432290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a775e80-89e5-425a-a3be-fd5123e1905b admin admin] PATH_INFO: `/v3/projects/08281572b8074b088f30be62b6fbade6/users/458aef1202f64c24a543017f69880fd7/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:49.450341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a775e80-89e5-425a-a3be-fd5123e1905b admin admin] RBAC: Authorizing `identity:create_grant(project_id=08281572b8074b088f30be62b6fbade6, user_id=458aef1202f64c24a543017f69880fd7, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:49.453204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a775e80-89e5-425a-a3be-fd5123e1905b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:49.566158 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1183/2397] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:06:49 2022] PUT /identity/v3/projects/08281572b8074b088f30be62b6fbade6/users/458aef1202f64c24a543017f69880fd7/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:06:49.567620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc97cfb-4993-4f4b-b871-20c2a98c4c0b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:49.592999 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc97cfb-4993-4f4b-b871-20c2a98c4c0b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:49.669645 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fc5bc-4b12-4469-8065-82ba0a3b0652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:49.670244 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fc5bc-4b12-4469-8065-82ba0a3b0652 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:49.670524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fc5bc-4b12-4469-8065-82ba0a3b0652 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:49.670800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fc5bc-4b12-4469-8065-82ba0a3b0652 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:49.671567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fc5bc-4b12-4469-8065-82ba0a3b0652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:49.673284 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fc5bc-4b12-4469-8065-82ba0a3b0652 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:49.687760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1215/2398] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:49.689278 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a775e80-89e5-425a-a3be-fd5123e1905b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:49.702747 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a775e80-89e5-425a-a3be-fd5123e1905b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:49.708284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14393fc4-062c-4d08-b9e5-f58de5c70031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:49.709063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14393fc4-062c-4d08-b9e5-f58de5c70031 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:49.709435 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14393fc4-062c-4d08-b9e5-f58de5c70031 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:49.709799 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14393fc4-062c-4d08-b9e5-f58de5c70031 admin admin] PATH_INFO: `/v3/projects/08281572b8074b088f30be62b6fbade6/users/458aef1202f64c24a543017f69880fd7/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:49.712898 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14393fc4-062c-4d08-b9e5-f58de5c70031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08281572b8074b088f30be62b6fbade6, user_id=458aef1202f64c24a543017f69880fd7, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:49.716185 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14393fc4-062c-4d08-b9e5-f58de5c70031 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:49.729092 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1184/2399] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:06:49 2022] PUT /identity/v3/projects/08281572b8074b088f30be62b6fbade6/users/458aef1202f64c24a543017f69880fd7/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:06:49.739792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964a606-9f24-4ecb-a830-5e141f6fb0ad None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:49.740080 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964a606-9f24-4ecb-a830-5e141f6fb0ad None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:49.740360 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964a606-9f24-4ecb-a830-5e141f6fb0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:49.776185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-9964a606-9f24-4ecb-a830-5e141f6fb0ad None None] MFA Rules not processed for user `458aef1202f64c24a543017f69880fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:49.777576 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9964a606-9f24-4ecb-a830-5e141f6fb0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:49.870662 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1216/2400] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:49.886040 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14393fc4-062c-4d08-b9e5-f58de5c70031 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:49.951733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14393fc4-062c-4d08-b9e5-f58de5c70031 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:49.956327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64056598-e42f-424c-847f-634e4ce59f94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:49.957156 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64056598-e42f-424c-847f-634e4ce59f94 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:49.957651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64056598-e42f-424c-847f-634e4ce59f94 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:49.958006 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64056598-e42f-424c-847f-634e4ce59f94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:50.030147 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:50.033532 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64056598-e42f-424c-847f-634e4ce59f94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:50.035149 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64056598-e42f-424c-847f-634e4ce59f94 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:50.045502 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1185/2401] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:53.713135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964a606-9f24-4ecb-a830-5e141f6fb0ad None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:53.717800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:53.793765 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964a606-9f24-4ecb-a830-5e141f6fb0ad None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:53.798694 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3edb58-de85-47c3-84a8-8688f989676a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:53.803026 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3edb58-de85-47c3-84a8-8688f989676a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:53.803026 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3edb58-de85-47c3-84a8-8688f989676a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:53.803026 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3edb58-de85-47c3-84a8-8688f989676a admin admin] PATH_INFO: `/v3/users/ed63a181838342a19137a4f04ff7bf9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:53.803026 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3edb58-de85-47c3-84a8-8688f989676a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed63a181838342a19137a4f04ff7bf9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:53.803468 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3edb58-de85-47c3-84a8-8688f989676a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:54.414653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ed3edb58-de85-47c3-84a8-8688f989676a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed63a181838342a19137a4f04ff7bf9a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:54.543727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ed3edb58-de85-47c3-84a8-8688f989676a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed63a181838342a19137a4f04ff7bf9a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:54.544439 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ed3edb58-de85-47c3-84a8-8688f989676a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed63a181838342a19137a4f04ff7bf9a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:54.552619 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ed3edb58-de85-47c3-84a8-8688f989676a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed63a181838342a19137a4f04ff7bf9a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:54.568189 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ed3edb58-de85-47c3-84a8-8688f989676a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed63a181838342a19137a4f04ff7bf9a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:54.575542 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1217/2402] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:06:53 2022] DELETE /identity/v3/users/ed63a181838342a19137a4f04ff7bf9a => generated 0 bytes in 863 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:54.580619 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc2ae6-555e-41c6-bda9-067153eab66a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:54.581037 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc2ae6-555e-41c6-bda9-067153eab66a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:54.581127 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc2ae6-555e-41c6-bda9-067153eab66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:54.613872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-6fdc2ae6-555e-41c6-bda9-067153eab66a None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:54.615840 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6fdc2ae6-555e-41c6-bda9-067153eab66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:54.692987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1186/2403] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:54.704235 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3edb58-de85-47c3-84a8-8688f989676a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:54.705695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ed3edb58-de85-47c3-84a8-8688f989676a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:54.783940 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3edb58-de85-47c3-84a8-8688f989676a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:54.788011 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8923e0-a192-4ffb-9f45-efe61e57d022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:54.788622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8923e0-a192-4ffb-9f45-efe61e57d022 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:54.788893 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8923e0-a192-4ffb-9f45-efe61e57d022 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:54.789172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8923e0-a192-4ffb-9f45-efe61e57d022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:54.803594 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8923e0-a192-4ffb-9f45-efe61e57d022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:54.805074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8923e0-a192-4ffb-9f45-efe61e57d022 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:54.818340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1218/2404] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:55.390146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc2ae6-555e-41c6-bda9-067153eab66a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:55.391714 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6fdc2ae6-555e-41c6-bda9-067153eab66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:55.435474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc2ae6-555e-41c6-bda9-067153eab66a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:55.446263 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86e0ec-3edf-4363-83bd-bf4afdc50808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:55.447530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86e0ec-3edf-4363-83bd-bf4afdc50808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:55.448089 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86e0ec-3edf-4363-83bd-bf4afdc50808 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:55.448713 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86e0ec-3edf-4363-83bd-bf4afdc50808 admin admin] PATH_INFO: `/v3/projects/f88c9d3db3cd472f8c7daaa9a0f27a24` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:55.452246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86e0ec-3edf-4363-83bd-bf4afdc50808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f88c9d3db3cd472f8c7daaa9a0f27a24)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:55.454501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86e0ec-3edf-4363-83bd-bf4afdc50808 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:55.594203 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4d86e0ec-3edf-4363-83bd-bf4afdc50808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f88c9d3db3cd472f8c7daaa9a0f27a24', 'request_id': 'req-4d86e0ec-3edf-4363-83bd-bf4afdc50808'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:55.606271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4d86e0ec-3edf-4363-83bd-bf4afdc50808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f88c9d3db3cd472f8c7daaa9a0f27a24', 'request_id': 'req-4d86e0ec-3edf-4363-83bd-bf4afdc50808'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:55.614976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4d86e0ec-3edf-4363-83bd-bf4afdc50808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f88c9d3db3cd472f8c7daaa9a0f27a24', 'request_id': 'req-4d86e0ec-3edf-4363-83bd-bf4afdc50808'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:55.738959 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4d86e0ec-3edf-4363-83bd-bf4afdc50808 admin admin] The token cache is being invalidate because project f88c9d3db3cd472f8c7daaa9a0f27a24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:06:55.739558 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4d86e0ec-3edf-4363-83bd-bf4afdc50808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:55.740409 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4d86e0ec-3edf-4363-83bd-bf4afdc50808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:55.745947 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1187/2405] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:06:55 2022] DELETE /identity/v3/projects/f88c9d3db3cd472f8c7daaa9a0f27a24 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:55.748220 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8923e0-a192-4ffb-9f45-efe61e57d022 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:55.760029 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8923e0-a192-4ffb-9f45-efe61e57d022 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:55.787610 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:55.844849 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75085c1-b785-4849-8218-82f967a54106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:55.847159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75085c1-b785-4849-8218-82f967a54106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:55.848303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75085c1-b785-4849-8218-82f967a54106 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:55.849754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75085c1-b785-4849-8218-82f967a54106 admin admin] PATH_INFO: `/v3/users/458aef1202f64c24a543017f69880fd7` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:55.851098 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75085c1-b785-4849-8218-82f967a54106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=458aef1202f64c24a543017f69880fd7)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:55.853714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75085c1-b785-4849-8218-82f967a54106 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:56.191602 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f75085c1-b785-4849-8218-82f967a54106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '458aef1202f64c24a543017f69880fd7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:56.303793 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f75085c1-b785-4849-8218-82f967a54106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '458aef1202f64c24a543017f69880fd7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:56.305082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f75085c1-b785-4849-8218-82f967a54106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '458aef1202f64c24a543017f69880fd7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:56.315746 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f75085c1-b785-4849-8218-82f967a54106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '458aef1202f64c24a543017f69880fd7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:56.347871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f75085c1-b785-4849-8218-82f967a54106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '458aef1202f64c24a543017f69880fd7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:56.356364 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1219/2406] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:06:55 2022] DELETE /identity/v3/users/458aef1202f64c24a543017f69880fd7 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:57.057715 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86e0ec-3edf-4363-83bd-bf4afdc50808 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:57.064509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4d86e0ec-3edf-4363-83bd-bf4afdc50808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:57.112430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86e0ec-3edf-4363-83bd-bf4afdc50808 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:57.199975 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472dfaf6-d000-44c4-b386-ab1cb406f205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:57.200937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472dfaf6-d000-44c4-b386-ab1cb406f205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:57.201295 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472dfaf6-d000-44c4-b386-ab1cb406f205 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:57.201651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472dfaf6-d000-44c4-b386-ab1cb406f205 admin admin] PATH_INFO: `/v3/projects/08281572b8074b088f30be62b6fbade6` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:57.203126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472dfaf6-d000-44c4-b386-ab1cb406f205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08281572b8074b088f30be62b6fbade6)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:57.205130 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472dfaf6-d000-44c4-b386-ab1cb406f205 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:57.238856 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:57.346024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-472dfaf6-d000-44c4-b386-ab1cb406f205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08281572b8074b088f30be62b6fbade6', 'request_id': 'req-472dfaf6-d000-44c4-b386-ab1cb406f205'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:57.352041 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-472dfaf6-d000-44c4-b386-ab1cb406f205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08281572b8074b088f30be62b6fbade6', 'request_id': 'req-472dfaf6-d000-44c4-b386-ab1cb406f205'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:57.359980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-472dfaf6-d000-44c4-b386-ab1cb406f205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08281572b8074b088f30be62b6fbade6', 'request_id': 'req-472dfaf6-d000-44c4-b386-ab1cb406f205'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:57.458811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-472dfaf6-d000-44c4-b386-ab1cb406f205 admin admin] The token cache is being invalidate because project 08281572b8074b088f30be62b6fbade6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:06:57.459441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-472dfaf6-d000-44c4-b386-ab1cb406f205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:57.460296 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-472dfaf6-d000-44c4-b386-ab1cb406f205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:06:57.462353 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1188/2407] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:06:57 2022] DELETE /identity/v3/projects/08281572b8074b088f30be62b6fbade6 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:06:57.467374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204092bc-9f8c-4349-ab3c-39082b35930d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:57.467903 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204092bc-9f8c-4349-ab3c-39082b35930d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:57.467903 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204092bc-9f8c-4349-ab3c-39082b35930d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:57.493511 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-204092bc-9f8c-4349-ab3c-39082b35930d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:57.495521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-204092bc-9f8c-4349-ab3c-39082b35930d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:57.564942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1220/2408] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:57.573831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba90c3-5195-49cc-bc25-e73bc97a5f80 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:57.574164 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba90c3-5195-49cc-bc25-e73bc97a5f80 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:57.574581 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba90c3-5195-49cc-bc25-e73bc97a5f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:57.597819 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a5ba90c3-5195-49cc-bc25-e73bc97a5f80 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:57.598952 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a5ba90c3-5195-49cc-bc25-e73bc97a5f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:57.607891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1189/2409] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:57.612727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204092bc-9f8c-4349-ab3c-39082b35930d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:57.624283 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204092bc-9f8c-4349-ab3c-39082b35930d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:57.630344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66d8b0-7e66-4f82-ac7f-85fff3dd7669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:57.631070 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66d8b0-7e66-4f82-ac7f-85fff3dd7669 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:57.631398 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66d8b0-7e66-4f82-ac7f-85fff3dd7669 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:57.631729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66d8b0-7e66-4f82-ac7f-85fff3dd7669 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:57.632541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66d8b0-7e66-4f82-ac7f-85fff3dd7669 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:06:57.632842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66d8b0-7e66-4f82-ac7f-85fff3dd7669 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:57.634762 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66d8b0-7e66-4f82-ac7f-85fff3dd7669 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:57.654019 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1221/2410] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:06:57.658942 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba90c3-5195-49cc-bc25-e73bc97a5f80 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:57.671414 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba90c3-5195-49cc-bc25-e73bc97a5f80 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:57.676781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2495c26-24e1-4238-8741-1f2c5603c38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:57.677444 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2495c26-24e1-4238-8741-1f2c5603c38f admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:57.677722 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2495c26-24e1-4238-8741-1f2c5603c38f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:57.677992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2495c26-24e1-4238-8741-1f2c5603c38f admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:57.678621 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2495c26-24e1-4238-8741-1f2c5603c38f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:57.680891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2495c26-24e1-4238-8741-1f2c5603c38f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:58.030797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1190/2411] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:06:57 2022] POST /identity/v3/projects => generated 371 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:06:58.035078 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66d8b0-7e66-4f82-ac7f-85fff3dd7669 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:58.045611 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66d8b0-7e66-4f82-ac7f-85fff3dd7669 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:58.076762 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:58.106307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dc02d-8fd5-41bd-ae02-7f50487e6b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:58.106961 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dc02d-8fd5-41bd-ae02-7f50487e6b26 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:58.107246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dc02d-8fd5-41bd-ae02-7f50487e6b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:58.107520 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dc02d-8fd5-41bd-ae02-7f50487e6b26 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:58.108163 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1dc02d-8fd5-41bd-ae02-7f50487e6b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:58.110237 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1dc02d-8fd5-41bd-ae02-7f50487e6b26 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:58.248941 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2495c26-24e1-4238-8741-1f2c5603c38f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:58.257007 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1222/2412] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:06:58 2022] POST /identity/v3/users => generated 286 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:06:58.265988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2495c26-24e1-4238-8741-1f2c5603c38f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:58.274873 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4f5ba-c00d-4f1e-9f75-9254facfc474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:58.277066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4f5ba-c00d-4f1e-9f75-9254facfc474 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:58.277587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4f5ba-c00d-4f1e-9f75-9254facfc474 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:58.277959 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4f5ba-c00d-4f1e-9f75-9254facfc474 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:58.278733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4f5ba-c00d-4f1e-9f75-9254facfc474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:58.280552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4f5ba-c00d-4f1e-9f75-9254facfc474 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:58.300603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dc02d-8fd5-41bd-ae02-7f50487e6b26 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:58.301738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1191/2413] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:06:58.317610 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dc02d-8fd5-41bd-ae02-7f50487e6b26 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:58.334400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c032e-1208-44f3-962e-f83ecbae541f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:58.335179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c032e-1208-44f3-962e-f83ecbae541f admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:58.335674 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c032e-1208-44f3-962e-f83ecbae541f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:58.338284 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c032e-1208-44f3-962e-f83ecbae541f admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:58.339077 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c032e-1208-44f3-962e-f83ecbae541f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:58.340829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c032e-1208-44f3-962e-f83ecbae541f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:58.357529 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1223/2414] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:06:58.367858 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4f5ba-c00d-4f1e-9f75-9254facfc474 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:58.386595 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4f5ba-c00d-4f1e-9f75-9254facfc474 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:58.395003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7328f9-19c8-4837-9ec1-f61de6928e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:58.395538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7328f9-19c8-4837-9ec1-f61de6928e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:58.395815 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7328f9-19c8-4837-9ec1-f61de6928e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:58.396086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7328f9-19c8-4837-9ec1-f61de6928e4b admin admin] PATH_INFO: `/v3/projects/64a6a624d46a4e6abaaaef497d955efc/users/5835060eae174e5c85d2ef7297431e66/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:58.416193 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7328f9-19c8-4837-9ec1-f61de6928e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a6a624d46a4e6abaaaef497d955efc, user_id=5835060eae174e5c85d2ef7297431e66, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:58.422483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7328f9-19c8-4837-9ec1-f61de6928e4b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:58.504862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1192/2415] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:06:58 2022] PUT /identity/v3/projects/64a6a624d46a4e6abaaaef497d955efc/users/5835060eae174e5c85d2ef7297431e66/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:06:58.516911 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed3874-1d75-4f1b-b71a-9cf3b7b6f384 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:58.522812 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed3874-1d75-4f1b-b71a-9cf3b7b6f384 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:58.522812 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed3874-1d75-4f1b-b71a-9cf3b7b6f384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:58.560146 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-9eed3874-1d75-4f1b-b71a-9cf3b7b6f384 None None] MFA Rules not processed for user `5835060eae174e5c85d2ef7297431e66`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:58.561896 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9eed3874-1d75-4f1b-b71a-9cf3b7b6f384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:58.642027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1224/2416] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:06:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:06:58.653718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7328f9-19c8-4837-9ec1-f61de6928e4b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:58.655584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1e7328f9-19c8-4837-9ec1-f61de6928e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:58.730385 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7328f9-19c8-4837-9ec1-f61de6928e4b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:58.734332 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d874da-ae49-481c-8b03-08ad69b61e9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:58.735047 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d874da-ae49-481c-8b03-08ad69b61e9b service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:58.735326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d874da-ae49-481c-8b03-08ad69b61e9b service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:58.735601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d874da-ae49-481c-8b03-08ad69b61e9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:58.744534 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d874da-ae49-481c-8b03-08ad69b61e9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:58.745980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d874da-ae49-481c-8b03-08ad69b61e9b service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:58.755634 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1193/2417] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:07:03.475808 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed3874-1d75-4f1b-b71a-9cf3b7b6f384 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:03.542782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed3874-1d75-4f1b-b71a-9cf3b7b6f384 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:03.547628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6190de-babe-43ee-b4ac-61046bbaff24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:03.548279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6190de-babe-43ee-b4ac-61046bbaff24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:03.548558 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6190de-babe-43ee-b4ac-61046bbaff24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:03.548836 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6190de-babe-43ee-b4ac-61046bbaff24 admin admin] PATH_INFO: `/v3/users/5835060eae174e5c85d2ef7297431e66` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:03.549998 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6190de-babe-43ee-b4ac-61046bbaff24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5835060eae174e5c85d2ef7297431e66)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:03.551990 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6190de-babe-43ee-b4ac-61046bbaff24 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:03.850391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:03.855502 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd6190de-babe-43ee-b4ac-61046bbaff24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5835060eae174e5c85d2ef7297431e66'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:03.983884 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd6190de-babe-43ee-b4ac-61046bbaff24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5835060eae174e5c85d2ef7297431e66'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:03.984404 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd6190de-babe-43ee-b4ac-61046bbaff24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5835060eae174e5c85d2ef7297431e66'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:03.992057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd6190de-babe-43ee-b4ac-61046bbaff24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5835060eae174e5c85d2ef7297431e66'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:04.005199 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd6190de-babe-43ee-b4ac-61046bbaff24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5835060eae174e5c85d2ef7297431e66'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:04.011066 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1225/2418] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:07:03 2022] DELETE /identity/v3/users/5835060eae174e5c85d2ef7297431e66 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:04.015678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb97ef-83fc-441c-b9c5-c83b3d8519d5 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:04.016159 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb97ef-83fc-441c-b9c5-c83b3d8519d5 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:04.016257 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb97ef-83fc-441c-b9c5-c83b3d8519d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:04.036433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-76eb97ef-83fc-441c-b9c5-c83b3d8519d5 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:04.037641 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-76eb97ef-83fc-441c-b9c5-c83b3d8519d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:04.083998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:04.099007 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1194/2419] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:07:04.109302 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6190de-babe-43ee-b4ac-61046bbaff24 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:04.110724 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-cd6190de-babe-43ee-b4ac-61046bbaff24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:04.169840 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6190de-babe-43ee-b4ac-61046bbaff24 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:04.173729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef3e81-97f9-42eb-8a7d-d9ab45396b21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:04.174324 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef3e81-97f9-42eb-8a7d-d9ab45396b21 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:04.174600 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef3e81-97f9-42eb-8a7d-d9ab45396b21 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:04.174865 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef3e81-97f9-42eb-8a7d-d9ab45396b21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:04.184638 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef3e81-97f9-42eb-8a7d-d9ab45396b21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:04.186059 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef3e81-97f9-42eb-8a7d-d9ab45396b21 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:04.195581 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1226/2420] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:07:04.670534 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb97ef-83fc-441c-b9c5-c83b3d8519d5 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:04.676381 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-76eb97ef-83fc-441c-b9c5-c83b3d8519d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:04.687145 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb97ef-83fc-441c-b9c5-c83b3d8519d5 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:04.692639 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7475c1-3643-4738-a5cf-9170751f536b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:04.693286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7475c1-3643-4738-a5cf-9170751f536b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:04.693592 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7475c1-3643-4738-a5cf-9170751f536b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:04.693946 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7475c1-3643-4738-a5cf-9170751f536b admin admin] PATH_INFO: `/v3/projects/64a6a624d46a4e6abaaaef497d955efc` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:04.695043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7475c1-3643-4738-a5cf-9170751f536b admin admin] RBAC: Authorizing `identity:delete_project(project_id=64a6a624d46a4e6abaaaef497d955efc)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:04.697173 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7475c1-3643-4738-a5cf-9170751f536b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:04.877955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1a7475c1-3643-4738-a5cf-9170751f536b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64a6a624d46a4e6abaaaef497d955efc', 'request_id': 'req-1a7475c1-3643-4738-a5cf-9170751f536b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:04.888188 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1a7475c1-3643-4738-a5cf-9170751f536b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64a6a624d46a4e6abaaaef497d955efc', 'request_id': 'req-1a7475c1-3643-4738-a5cf-9170751f536b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:04.895931 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1a7475c1-3643-4738-a5cf-9170751f536b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64a6a624d46a4e6abaaaef497d955efc', 'request_id': 'req-1a7475c1-3643-4738-a5cf-9170751f536b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:05.034497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1a7475c1-3643-4738-a5cf-9170751f536b admin admin] The token cache is being invalidate because project 64a6a624d46a4e6abaaaef497d955efc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:07:05.035558 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1a7475c1-3643-4738-a5cf-9170751f536b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:05.036236 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1a7475c1-3643-4738-a5cf-9170751f536b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:05.038066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1195/2421] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:07:04 2022] DELETE /identity/v3/projects/64a6a624d46a4e6abaaaef497d955efc => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:05.043106 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d270998-af93-4d62-841b-88d5341e92fd None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:05.043569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d270998-af93-4d62-841b-88d5341e92fd None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:05.043685 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d270998-af93-4d62-841b-88d5341e92fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:05.066628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-7d270998-af93-4d62-841b-88d5341e92fd None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:05.067948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7d270998-af93-4d62-841b-88d5341e92fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:05.132181 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1227/2422] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:07:05.141842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61866fe3-79f4-4841-8930-161e60c84ba7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:05.142128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61866fe3-79f4-4841-8930-161e60c84ba7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:05.142407 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61866fe3-79f4-4841-8930-161e60c84ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:05.163498 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-61866fe3-79f4-4841-8930-161e60c84ba7 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:05.164705 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-61866fe3-79f4-4841-8930-161e60c84ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:05.172953 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1196/2423] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:07:05.177327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d270998-af93-4d62-841b-88d5341e92fd None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:05.187761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d270998-af93-4d62-841b-88d5341e92fd None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:05.192402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081bf724-d776-4f9e-b70f-0d1dd58e8e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:05.195666 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bf724-d776-4f9e-b70f-0d1dd58e8e24 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:05.195666 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bf724-d776-4f9e-b70f-0d1dd58e8e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:05.195666 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bf724-d776-4f9e-b70f-0d1dd58e8e24 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:05.195666 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081bf724-d776-4f9e-b70f-0d1dd58e8e24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:07:05.195666 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081bf724-d776-4f9e-b70f-0d1dd58e8e24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:05.196163 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081bf724-d776-4f9e-b70f-0d1dd58e8e24 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:05.208281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1228/2424] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:07:05.212935 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61866fe3-79f4-4841-8930-161e60c84ba7 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:05.224428 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61866fe3-79f4-4841-8930-161e60c84ba7 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:05.233159 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b0237-4905-462c-99d7-bb6d6bf0a744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:05.233159 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b0237-4905-462c-99d7-bb6d6bf0a744 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:05.233159 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b0237-4905-462c-99d7-bb6d6bf0a744 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:05.233159 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b0237-4905-462c-99d7-bb6d6bf0a744 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:05.233930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b0237-4905-462c-99d7-bb6d6bf0a744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:05.233930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b0237-4905-462c-99d7-bb6d6bf0a744 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:05.395852 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1197/2425] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:07:05 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:07:05.397122 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081bf724-d776-4f9e-b70f-0d1dd58e8e24 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:05.408064 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081bf724-d776-4f9e-b70f-0d1dd58e8e24 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:05.459612 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:05.471604 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8a606-9991-4723-af9c-df8307dec2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:05.472236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8a606-9991-4723-af9c-df8307dec2c8 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:05.472516 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8a606-9991-4723-af9c-df8307dec2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:05.472789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8a606-9991-4723-af9c-df8307dec2c8 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:05.473431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8a606-9991-4723-af9c-df8307dec2c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:05.480746 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8a606-9991-4723-af9c-df8307dec2c8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:05.562516 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1229/2426] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:07:05 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:07:05.566753 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b0237-4905-462c-99d7-bb6d6bf0a744 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:05.577098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b0237-4905-462c-99d7-bb6d6bf0a744 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:05.581816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca83839-fb02-46b0-a501-bebd9062b1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:05.582406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca83839-fb02-46b0-a501-bebd9062b1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:05.582686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca83839-fb02-46b0-a501-bebd9062b1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:05.582989 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca83839-fb02-46b0-a501-bebd9062b1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:05.583625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca83839-fb02-46b0-a501-bebd9062b1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:05.585329 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca83839-fb02-46b0-a501-bebd9062b1b8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:05.595601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1198/2427] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:07:05.599788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8a606-9991-4723-af9c-df8307dec2c8 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:05.611700 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8a606-9991-4723-af9c-df8307dec2c8 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:05.616543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490e9ca-21c3-42c8-a379-fc2443aeece8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:05.617403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490e9ca-21c3-42c8-a379-fc2443aeece8 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:05.617687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490e9ca-21c3-42c8-a379-fc2443aeece8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:05.617959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490e9ca-21c3-42c8-a379-fc2443aeece8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:05.618607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490e9ca-21c3-42c8-a379-fc2443aeece8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:05.621166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490e9ca-21c3-42c8-a379-fc2443aeece8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:05.635970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1230/2428] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:07:05.640883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca83839-fb02-46b0-a501-bebd9062b1b8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:05.656728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca83839-fb02-46b0-a501-bebd9062b1b8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:05.665666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3623e69-9297-414d-b96f-f330f5cf70e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:05.666828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3623e69-9297-414d-b96f-f330f5cf70e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:05.667134 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3623e69-9297-414d-b96f-f330f5cf70e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:05.667405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3623e69-9297-414d-b96f-f330f5cf70e2 admin admin] PATH_INFO: `/v3/projects/fefb0f83ce4447cc9223c383a256676a/users/b4cc11524fe542c5a5361b3630e6d3a6/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:05.685433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3623e69-9297-414d-b96f-f330f5cf70e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fefb0f83ce4447cc9223c383a256676a, user_id=b4cc11524fe542c5a5361b3630e6d3a6, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:05.688289 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3623e69-9297-414d-b96f-f330f5cf70e2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:05.758578 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1199/2429] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:07:05 2022] PUT /identity/v3/projects/fefb0f83ce4447cc9223c383a256676a/users/b4cc11524fe542c5a5361b3630e6d3a6/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:05.764019 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf68c5-9d48-47bb-a66b-a676f22b5363 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:05.764277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf68c5-9d48-47bb-a66b-a676f22b5363 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:05.764547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf68c5-9d48-47bb-a66b-a676f22b5363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:05.784156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-36cf68c5-9d48-47bb-a66b-a676f22b5363 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:05.785344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-36cf68c5-9d48-47bb-a66b-a676f22b5363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:05.845634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1231/2430] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:07:05.856096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3623e69-9297-414d-b96f-f330f5cf70e2 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:05.857784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c3623e69-9297-414d-b96f-f330f5cf70e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:05.921274 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3623e69-9297-414d-b96f-f330f5cf70e2 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:05.925059 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb71ee-2d0b-4247-9abc-fc7e6e974c95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:05.925683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb71ee-2d0b-4247-9abc-fc7e6e974c95 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:05.925941 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb71ee-2d0b-4247-9abc-fc7e6e974c95 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:05.926213 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb71ee-2d0b-4247-9abc-fc7e6e974c95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:05.935398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb71ee-2d0b-4247-9abc-fc7e6e974c95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:05.936998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb71ee-2d0b-4247-9abc-fc7e6e974c95 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:05.946490 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1200/2431] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:07:07.589733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f89f6-3ea3-4770-a6aa-81e3a5ce401f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:07.590322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f89f6-3ea3-4770-a6aa-81e3a5ce401f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:07.590322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f89f6-3ea3-4770-a6aa-81e3a5ce401f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:07.622249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1e8f89f6-3ea3-4770-a6aa-81e3a5ce401f None None] MFA Rules not processed for user `b4cc11524fe542c5a5361b3630e6d3a6`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:07.623530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1e8f89f6-3ea3-4770-a6aa-81e3a5ce401f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:07.693392 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1232/2432] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:07:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:07:07.704671 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb71ee-2d0b-4247-9abc-fc7e6e974c95 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:07.706100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-75fb71ee-2d0b-4247-9abc-fc7e6e974c95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:07.720168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:07.771990 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb71ee-2d0b-4247-9abc-fc7e6e974c95 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:07.777441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9938272-6154-43d2-9f7c-b9b251cb2430 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:07.778050 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9938272-6154-43d2-9f7c-b9b251cb2430 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:07.778351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9938272-6154-43d2-9f7c-b9b251cb2430 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:07.778667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9938272-6154-43d2-9f7c-b9b251cb2430 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:07.787229 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9938272-6154-43d2-9f7c-b9b251cb2430 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:07.788776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9938272-6154-43d2-9f7c-b9b251cb2430 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:07.798752 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1201/2433] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:07:09.690291 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0bf87-98dd-4bf3-9613-39642b603967 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:09.690881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0bf87-98dd-4bf3-9613-39642b603967 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:09.690881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0bf87-98dd-4bf3-9613-39642b603967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:09.710817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-41c0bf87-98dd-4bf3-9613-39642b603967 None None] MFA Rules not processed for user `b4cc11524fe542c5a5361b3630e6d3a6`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:09.712132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-41c0bf87-98dd-4bf3-9613-39642b603967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:09.719067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1233/2434] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:07:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:07:10.509413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5e422-a6b2-429a-8068-290fbd41cc82 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:10.509941 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5e422-a6b2-429a-8068-290fbd41cc82 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:10.510027 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5e422-a6b2-429a-8068-290fbd41cc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:10.550308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-94f5e422-a6b2-429a-8068-290fbd41cc82 None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:10.553897 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-94f5e422-a6b2-429a-8068-290fbd41cc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:10.668126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1202/2435] 10.222.0.91 () {58 vars in 1009 bytes} [Thu Sep 8 14:07:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:07:10.682687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0bf87-98dd-4bf3-9613-39642b603967 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:10.682854 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-41c0bf87-98dd-4bf3-9613-39642b603967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:10.736668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:10.754699 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0bf87-98dd-4bf3-9613-39642b603967 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:10.760769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322a94e-b01c-4834-ad14-8501d8839482 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:10.766268 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322a94e-b01c-4834-ad14-8501d8839482 service swift] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:10.766268 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322a94e-b01c-4834-ad14-8501d8839482 service swift] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:10.766268 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322a94e-b01c-4834-ad14-8501d8839482 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:10.772566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322a94e-b01c-4834-ad14-8501d8839482 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:10.774523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1322a94e-b01c-4834-ad14-8501d8839482 service swift] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:10.785112 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1234/2436] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 14:07:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:07:20.647980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=46 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:21.095908 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=9 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:31.390554 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef29bb-2cf1-408e-9d75-a68a235ea1ef None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:31.391514 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef29bb-2cf1-408e-9d75-a68a235ea1ef None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:31.391945 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef29bb-2cf1-408e-9d75-a68a235ea1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:31.415491 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-78ef29bb-2cf1-408e-9d75-a68a235ea1ef None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:31.418367 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-78ef29bb-2cf1-408e-9d75-a68a235ea1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:31.439192 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1203/2437] 10.222.0.91 () {58 vars in 967 bytes} [Thu Sep 8 14:07:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:07:31.444413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322a94e-b01c-4834-ad14-8501d8839482 service swift] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:31.472625 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1322a94e-b01c-4834-ad14-8501d8839482 service swift] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:31.486840 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d02b9-43f9-494d-b51d-f5a9eb16eee7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:31.488602 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d02b9-43f9-494d-b51d-f5a9eb16eee7 service swift] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:31.489040 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d02b9-43f9-494d-b51d-f5a9eb16eee7 service swift] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:31.489753 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d02b9-43f9-494d-b51d-f5a9eb16eee7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:31.504926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d02b9-43f9-494d-b51d-f5a9eb16eee7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:31.506685 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d02b9-43f9-494d-b51d-f5a9eb16eee7 service swift] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:31.516377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1235/2438] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 14:07:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:07:41.373071 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef29bb-2cf1-408e-9d75-a68a235ea1ef None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:41.374525 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-78ef29bb-2cf1-408e-9d75-a68a235ea1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:41.386994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef29bb-2cf1-408e-9d75-a68a235ea1ef None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:41.390696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be136f-df03-420b-8f5f-279c86c6fef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:41.391401 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be136f-df03-420b-8f5f-279c86c6fef6 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:41.391677 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be136f-df03-420b-8f5f-279c86c6fef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:41.391948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be136f-df03-420b-8f5f-279c86c6fef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:41.393672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-62be136f-df03-420b-8f5f-279c86c6fef6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:41.406287 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be136f-df03-420b-8f5f-279c86c6fef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:41.407774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be136f-df03-420b-8f5f-279c86c6fef6 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:41.418029 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1204/2439] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:07:41.513210 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=6 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:43.378425 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d02b9-43f9-494d-b51d-f5a9eb16eee7 service swift] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:43.380143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-230d02b9-43f9-494d-b51d-f5a9eb16eee7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:43.395056 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d02b9-43f9-494d-b51d-f5a9eb16eee7 service swift] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:43.399919 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd493f-b466-4541-9105-6a99a6d57671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:43.400538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd493f-b466-4541-9105-6a99a6d57671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:43.400843 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd493f-b466-4541-9105-6a99a6d57671 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:43.401116 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd493f-b466-4541-9105-6a99a6d57671 admin admin] PATH_INFO: `/v3/users/29040802b064433f83aee605e0b496c2` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:43.402274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd493f-b466-4541-9105-6a99a6d57671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29040802b064433f83aee605e0b496c2)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:43.404159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd493f-b466-4541-9105-6a99a6d57671 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:43.615236 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-25bd493f-b466-4541-9105-6a99a6d57671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29040802b064433f83aee605e0b496c2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:43.692327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-25bd493f-b466-4541-9105-6a99a6d57671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29040802b064433f83aee605e0b496c2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:43.693391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-25bd493f-b466-4541-9105-6a99a6d57671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29040802b064433f83aee605e0b496c2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:43.703385 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-25bd493f-b466-4541-9105-6a99a6d57671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29040802b064433f83aee605e0b496c2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:43.718689 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-25bd493f-b466-4541-9105-6a99a6d57671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29040802b064433f83aee605e0b496c2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:43.728517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1236/2440] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:07:43 2022] DELETE /identity/v3/users/29040802b064433f83aee605e0b496c2 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:43.735686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e80a5-618e-48e5-b888-26408e4e3f68 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:43.736125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e80a5-618e-48e5-b888-26408e4e3f68 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:43.736289 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e80a5-618e-48e5-b888-26408e4e3f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:43.755976 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b10e80a5-618e-48e5-b888-26408e4e3f68 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:43.757172 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b10e80a5-618e-48e5-b888-26408e4e3f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:43.818209 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1205/2441] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:07:43.828279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd493f-b466-4541-9105-6a99a6d57671 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:43.829697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-25bd493f-b466-4541-9105-6a99a6d57671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:43.894141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd493f-b466-4541-9105-6a99a6d57671 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:43.898146 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cc9fc-3580-4e21-86eb-bc910805b5ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:43.899048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cc9fc-3580-4e21-86eb-bc910805b5ea service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:43.899327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cc9fc-3580-4e21-86eb-bc910805b5ea service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:43.899600 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cc9fc-3580-4e21-86eb-bc910805b5ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:43.909501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8cc9fc-3580-4e21-86eb-bc910805b5ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:43.910986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8cc9fc-3580-4e21-86eb-bc910805b5ea service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:43.925728 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1237/2442] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:07:44.493279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e80a5-618e-48e5-b888-26408e4e3f68 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:44.494706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b10e80a5-618e-48e5-b888-26408e4e3f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:44.506847 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e80a5-618e-48e5-b888-26408e4e3f68 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:44.513110 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71f3bc-360a-4520-b559-cf6401169f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:44.513819 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71f3bc-360a-4520-b559-cf6401169f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:44.514112 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71f3bc-360a-4520-b559-cf6401169f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:44.514850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71f3bc-360a-4520-b559-cf6401169f54 admin admin] PATH_INFO: `/v3/projects/f33dbf40d3e24e1f934b25abe0253ea9` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:44.525119 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71f3bc-360a-4520-b559-cf6401169f54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f33dbf40d3e24e1f934b25abe0253ea9)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:44.525838 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b71f3bc-360a-4520-b559-cf6401169f54 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:44.649305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3b71f3bc-360a-4520-b559-cf6401169f54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f33dbf40d3e24e1f934b25abe0253ea9', 'request_id': 'req-3b71f3bc-360a-4520-b559-cf6401169f54'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:44.662624 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3b71f3bc-360a-4520-b559-cf6401169f54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f33dbf40d3e24e1f934b25abe0253ea9', 'request_id': 'req-3b71f3bc-360a-4520-b559-cf6401169f54'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:44.671716 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3b71f3bc-360a-4520-b559-cf6401169f54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f33dbf40d3e24e1f934b25abe0253ea9', 'request_id': 'req-3b71f3bc-360a-4520-b559-cf6401169f54'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:44.753792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3b71f3bc-360a-4520-b559-cf6401169f54 admin admin] The token cache is being invalidate because project f33dbf40d3e24e1f934b25abe0253ea9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:07:44.754431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3b71f3bc-360a-4520-b559-cf6401169f54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:44.754738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3b71f3bc-360a-4520-b559-cf6401169f54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:44.756477 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1206/2443] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:07:44 2022] DELETE /identity/v3/projects/f33dbf40d3e24e1f934b25abe0253ea9 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:44.762501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cc9fc-3580-4e21-86eb-bc910805b5ea service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:44.773034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cc9fc-3580-4e21-86eb-bc910805b5ea service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:44.849373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c3709-dc33-4ba2-b52f-3efb8d011892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:44.850028 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c3709-dc33-4ba2-b52f-3efb8d011892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:44.850305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c3709-dc33-4ba2-b52f-3efb8d011892 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:44.850740 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c3709-dc33-4ba2-b52f-3efb8d011892 admin admin] PATH_INFO: `/v3/users/07583736988b4d119befe6145f4b8172` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:44.852182 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c3709-dc33-4ba2-b52f-3efb8d011892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07583736988b4d119befe6145f4b8172)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:44.854387 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c3709-dc33-4ba2-b52f-3efb8d011892 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:44.878711 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:45.060223 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c41c3709-dc33-4ba2-b52f-3efb8d011892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07583736988b4d119befe6145f4b8172'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:45.128949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c41c3709-dc33-4ba2-b52f-3efb8d011892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07583736988b4d119befe6145f4b8172'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:45.131191 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c41c3709-dc33-4ba2-b52f-3efb8d011892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07583736988b4d119befe6145f4b8172'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:45.148136 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c41c3709-dc33-4ba2-b52f-3efb8d011892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07583736988b4d119befe6145f4b8172'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:45.161591 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c41c3709-dc33-4ba2-b52f-3efb8d011892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07583736988b4d119befe6145f4b8172'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:45.168605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1238/2444] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:07:44 2022] DELETE /identity/v3/users/07583736988b4d119befe6145f4b8172 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:45.657938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71f3bc-360a-4520-b559-cf6401169f54 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:45.660098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3b71f3bc-360a-4520-b559-cf6401169f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:45.698807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71f3bc-360a-4520-b559-cf6401169f54 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:45.770472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c3709-dc33-4ba2-b52f-3efb8d011892 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:45.772578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c41c3709-dc33-4ba2-b52f-3efb8d011892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:45.779364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9e1e4-8380-4d67-9d4d-e96b569392f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:45.780271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9e1e4-8380-4d67-9d4d-e96b569392f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:45.780732 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9e1e4-8380-4d67-9d4d-e96b569392f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:45.781069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9e1e4-8380-4d67-9d4d-e96b569392f9 admin admin] PATH_INFO: `/v3/users/beb41eb66e8140a6999b0bddab3c4afd` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:45.787687 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9e1e4-8380-4d67-9d4d-e96b569392f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beb41eb66e8140a6999b0bddab3c4afd)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:45.789691 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c3709-dc33-4ba2-b52f-3efb8d011892 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:45.790791 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9e1e4-8380-4d67-9d4d-e96b569392f9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:45.798056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:45.804914 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f446e-200b-4eba-ac2f-b9316a12f6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:45.806531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f446e-200b-4eba-ac2f-b9316a12f6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:45.807131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f446e-200b-4eba-ac2f-b9316a12f6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:45.807954 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f446e-200b-4eba-ac2f-b9316a12f6a9 admin admin] PATH_INFO: `/v3/projects/a8d43bd71c744641835b6bd91d9d755e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:45.809276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f446e-200b-4eba-ac2f-b9316a12f6a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8d43bd71c744641835b6bd91d9d755e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:45.811453 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f446e-200b-4eba-ac2f-b9316a12f6a9 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:46.003869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e2f446e-200b-4eba-ac2f-b9316a12f6a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d43bd71c744641835b6bd91d9d755e', 'request_id': 'req-2e2f446e-200b-4eba-ac2f-b9316a12f6a9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:46.021107 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dad9e1e4-8380-4d67-9d4d-e96b569392f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beb41eb66e8140a6999b0bddab3c4afd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:46.097847 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e2f446e-200b-4eba-ac2f-b9316a12f6a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d43bd71c744641835b6bd91d9d755e', 'request_id': 'req-2e2f446e-200b-4eba-ac2f-b9316a12f6a9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:46.104895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e2f446e-200b-4eba-ac2f-b9316a12f6a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d43bd71c744641835b6bd91d9d755e', 'request_id': 'req-2e2f446e-200b-4eba-ac2f-b9316a12f6a9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:46.117385 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e2f446e-200b-4eba-ac2f-b9316a12f6a9 admin admin] The token cache is being invalidate because project a8d43bd71c744641835b6bd91d9d755e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:07:46.117717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e2f446e-200b-4eba-ac2f-b9316a12f6a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:46.118445 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e2f446e-200b-4eba-ac2f-b9316a12f6a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:46.120130 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1239/2445] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:07:45 2022] DELETE /identity/v3/projects/a8d43bd71c744641835b6bd91d9d755e => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:07:46.126088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa83cd0-b000-4113-8570-e029858f14d8 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:46.126377 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa83cd0-b000-4113-8570-e029858f14d8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:46.126654 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa83cd0-b000-4113-8570-e029858f14d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:46.155889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dad9e1e4-8380-4d67-9d4d-e96b569392f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beb41eb66e8140a6999b0bddab3c4afd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:46.170133 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-7fa83cd0-b000-4113-8570-e029858f14d8 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:46.170937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dad9e1e4-8380-4d67-9d4d-e96b569392f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beb41eb66e8140a6999b0bddab3c4afd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:46.171393 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7fa83cd0-b000-4113-8570-e029858f14d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:46.171833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dad9e1e4-8380-4d67-9d4d-e96b569392f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beb41eb66e8140a6999b0bddab3c4afd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:46.179873 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dad9e1e4-8380-4d67-9d4d-e96b569392f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beb41eb66e8140a6999b0bddab3c4afd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:46.187008 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1207/2446] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:07:45 2022] DELETE /identity/v3/users/beb41eb66e8140a6999b0bddab3c4afd => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:46.233061 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1240/2447] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:07:46.242180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bd507-3267-4525-8d9c-e18bbc4d389b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:46.242540 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bd507-3267-4525-8d9c-e18bbc4d389b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:46.242829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bd507-3267-4525-8d9c-e18bbc4d389b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:46.263990 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-fd0bd507-3267-4525-8d9c-e18bbc4d389b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:46.265769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fd0bd507-3267-4525-8d9c-e18bbc4d389b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:46.331436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1208/2448] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:07:46.335987 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa83cd0-b000-4113-8570-e029858f14d8 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:46.341684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:46.347502 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa83cd0-b000-4113-8570-e029858f14d8 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:46.352318 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff633969-70b6-45ec-990b-6134b0c9f9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:46.352945 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff633969-70b6-45ec-990b-6134b0c9f9fb admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:46.353229 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff633969-70b6-45ec-990b-6134b0c9f9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:46.353467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff633969-70b6-45ec-990b-6134b0c9f9fb admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:46.354108 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff633969-70b6-45ec-990b-6134b0c9f9fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:07:46.354408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff633969-70b6-45ec-990b-6134b0c9f9fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:46.356777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff633969-70b6-45ec-990b-6134b0c9f9fb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:46.368667 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1241/2449] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:07:46.375938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0bd507-3267-4525-8d9c-e18bbc4d389b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:46.383198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0bd507-3267-4525-8d9c-e18bbc4d389b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:46.388093 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86fd58-cc22-4c8c-baee-a7cdc50dc7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:46.388693 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86fd58-cc22-4c8c-baee-a7cdc50dc7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:46.388979 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86fd58-cc22-4c8c-baee-a7cdc50dc7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:46.389253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86fd58-cc22-4c8c-baee-a7cdc50dc7b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:46.389916 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86fd58-cc22-4c8c-baee-a7cdc50dc7b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:46.391873 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86fd58-cc22-4c8c-baee-a7cdc50dc7b6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:46.489488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff633969-70b6-45ec-990b-6134b0c9f9fb admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:46.491327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ff633969-70b6-45ec-990b-6134b0c9f9fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:46.505548 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff633969-70b6-45ec-990b-6134b0c9f9fb admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:46.510835 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b4f14-21c5-40a1-8bcb-aba9d52ef69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:46.511499 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b4f14-21c5-40a1-8bcb-aba9d52ef69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:46.519621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b4f14-21c5-40a1-8bcb-aba9d52ef69a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:46.519621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b4f14-21c5-40a1-8bcb-aba9d52ef69a admin admin] PATH_INFO: `/v3/projects/681592d8b63a438c8b4893eca9a3f04b` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:46.519621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b4f14-21c5-40a1-8bcb-aba9d52ef69a admin admin] RBAC: Authorizing `identity:delete_project(project_id=681592d8b63a438c8b4893eca9a3f04b)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:46.519621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b4f14-21c5-40a1-8bcb-aba9d52ef69a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:46.526335 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1209/2450] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:07:46 2022] POST /identity/v3/projects => generated 365 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:07:46.530956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86fd58-cc22-4c8c-baee-a7cdc50dc7b6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:46.597705 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86fd58-cc22-4c8c-baee-a7cdc50dc7b6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:46.602245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c2c09-699e-42f3-9cd6-f260a737757f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:46.602878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c2c09-699e-42f3-9cd6-f260a737757f admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:46.603189 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c2c09-699e-42f3-9cd6-f260a737757f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:46.603458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473c2c09-699e-42f3-9cd6-f260a737757f admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:46.604091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c2c09-699e-42f3-9cd6-f260a737757f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:46.606670 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473c2c09-699e-42f3-9cd6-f260a737757f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:46.615707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:46.633389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-781b4f14-21c5-40a1-8bcb-aba9d52ef69a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '681592d8b63a438c8b4893eca9a3f04b', 'request_id': 'req-781b4f14-21c5-40a1-8bcb-aba9d52ef69a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:46.711944 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-781b4f14-21c5-40a1-8bcb-aba9d52ef69a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '681592d8b63a438c8b4893eca9a3f04b', 'request_id': 'req-781b4f14-21c5-40a1-8bcb-aba9d52ef69a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:46.723307 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1210/2451] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:07:46 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:07:46.723541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-781b4f14-21c5-40a1-8bcb-aba9d52ef69a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '681592d8b63a438c8b4893eca9a3f04b', 'request_id': 'req-781b4f14-21c5-40a1-8bcb-aba9d52ef69a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:46.724405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c2c09-699e-42f3-9cd6-f260a737757f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:46.734181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473c2c09-699e-42f3-9cd6-f260a737757f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:46.734693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-781b4f14-21c5-40a1-8bcb-aba9d52ef69a admin admin] The token cache is being invalidate because project 681592d8b63a438c8b4893eca9a3f04b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:07:46.735146 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-781b4f14-21c5-40a1-8bcb-aba9d52ef69a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:46.735869 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-781b4f14-21c5-40a1-8bcb-aba9d52ef69a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:46.737391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1242/2452] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:07:46 2022] DELETE /identity/v3/projects/681592d8b63a438c8b4893eca9a3f04b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:07:46.742212 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b1178-aef0-47a8-a277-21315d61c5fc None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:46.742524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b1178-aef0-47a8-a277-21315d61c5fc None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:46.742865 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b1178-aef0-47a8-a277-21315d61c5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:46.762861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-084b1178-aef0-47a8-a277-21315d61c5fc None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:46.764246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-084b1178-aef0-47a8-a277-21315d61c5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:46.797479 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b464a50-d5d2-4a7d-be68-9a10faacc7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:46.798069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b464a50-d5d2-4a7d-be68-9a10faacc7e2 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:46.798358 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b464a50-d5d2-4a7d-be68-9a10faacc7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:46.798628 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b464a50-d5d2-4a7d-be68-9a10faacc7e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:46.799368 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b464a50-d5d2-4a7d-be68-9a10faacc7e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:46.800998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b464a50-d5d2-4a7d-be68-9a10faacc7e2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:46.815404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1211/2453] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:07:46.819165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b464a50-d5d2-4a7d-be68-9a10faacc7e2 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:46.820884 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5b464a50-d5d2-4a7d-be68-9a10faacc7e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:46.835109 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b464a50-d5d2-4a7d-be68-9a10faacc7e2 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:46.837876 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1243/2454] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:07:46.847906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd3d71-ee32-4de6-9d9d-f1290acad55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:46.847906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd3d71-ee32-4de6-9d9d-f1290acad55e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:46.847906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd3d71-ee32-4de6-9d9d-f1290acad55e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:46.847906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd3d71-ee32-4de6-9d9d-f1290acad55e admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:46.848413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd3d71-ee32-4de6-9d9d-f1290acad55e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:46.848413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd3d71-ee32-4de6-9d9d-f1290acad55e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:46.849188 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687dbc32-367c-4976-8a5d-a52febfd9321 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:46.849560 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687dbc32-367c-4976-8a5d-a52febfd9321 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:46.850029 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687dbc32-367c-4976-8a5d-a52febfd9321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:46.859120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1212/2455] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:07:46.860937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd3d71-ee32-4de6-9d9d-f1290acad55e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:46.871352 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd3d71-ee32-4de6-9d9d-f1290acad55e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:46.876396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-687dbc32-367c-4976-8a5d-a52febfd9321 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:46.876851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386150c-ab84-4462-89d9-47e442b77c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:46.878702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386150c-ab84-4462-89d9-47e442b77c44 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:46.879227 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-687dbc32-367c-4976-8a5d-a52febfd9321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:46.879324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386150c-ab84-4462-89d9-47e442b77c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:46.879324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386150c-ab84-4462-89d9-47e442b77c44 admin admin] PATH_INFO: `/v3/projects/ddacd36c7ec04dd0bb485e4b32162a6f/users/566ff7aaedc1402e98ce303813003aa4/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:46.889475 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1244/2456] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:07:46.895034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687dbc32-367c-4976-8a5d-a52febfd9321 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:46.913598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386150c-ab84-4462-89d9-47e442b77c44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddacd36c7ec04dd0bb485e4b32162a6f, user_id=566ff7aaedc1402e98ce303813003aa4, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:46.916544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386150c-ab84-4462-89d9-47e442b77c44 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:46.919396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687dbc32-367c-4976-8a5d-a52febfd9321 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:46.924879 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547befc-1c5d-43fb-ad20-c2d40f30fb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:46.925572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547befc-1c5d-43fb-ad20-c2d40f30fb54 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:46.925974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547befc-1c5d-43fb-ad20-c2d40f30fb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:46.926312 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547befc-1c5d-43fb-ad20-c2d40f30fb54 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:46.927144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c547befc-1c5d-43fb-ad20-c2d40f30fb54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:07:46.927532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c547befc-1c5d-43fb-ad20-c2d40f30fb54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:46.929401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c547befc-1c5d-43fb-ad20-c2d40f30fb54 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:46.943476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1245/2457] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:07:46.949551 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547befc-1c5d-43fb-ad20-c2d40f30fb54 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:46.959936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547befc-1c5d-43fb-ad20-c2d40f30fb54 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:46.965281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db0332-1aec-431c-9aff-81081fd6bf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:46.965867 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db0332-1aec-431c-9aff-81081fd6bf97 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:46.966149 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db0332-1aec-431c-9aff-81081fd6bf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:46.966427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db0332-1aec-431c-9aff-81081fd6bf97 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:46.967258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db0332-1aec-431c-9aff-81081fd6bf97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:46.969075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db0332-1aec-431c-9aff-81081fd6bf97 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:47.024222 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1213/2458] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:07:46 2022] PUT /identity/v3/projects/ddacd36c7ec04dd0bb485e4b32162a6f/users/566ff7aaedc1402e98ce303813003aa4/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:47.035628 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4501971-6e92-40e7-8f66-c2f0e2f08252 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:47.039907 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4501971-6e92-40e7-8f66-c2f0e2f08252 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:47.039907 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4501971-6e92-40e7-8f66-c2f0e2f08252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:47.066351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b4501971-6e92-40e7-8f66-c2f0e2f08252 None None] MFA Rules not processed for user `566ff7aaedc1402e98ce303813003aa4`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:47.070615 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1246/2459] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:07:46 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:07:47.070774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b4501971-6e92-40e7-8f66-c2f0e2f08252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:47.072307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db0332-1aec-431c-9aff-81081fd6bf97 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:47.108513 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:47.138403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db0332-1aec-431c-9aff-81081fd6bf97 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:47.140905 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1214/2460] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:07:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:07:47.143533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807a9b9-bd07-4edf-bef7-0d99e32de221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:47.144405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807a9b9-bd07-4edf-bef7-0d99e32de221 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:47.144799 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807a9b9-bd07-4edf-bef7-0d99e32de221 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:47.145056 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807a9b9-bd07-4edf-bef7-0d99e32de221 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:47.145285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4501971-6e92-40e7-8f66-c2f0e2f08252 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:47.145686 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807a9b9-bd07-4edf-bef7-0d99e32de221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:47.147684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807a9b9-bd07-4edf-bef7-0d99e32de221 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:47.156408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4501971-6e92-40e7-8f66-c2f0e2f08252 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:47.161362 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e5155-a895-4dc8-8c5b-bf860ac8a3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:47.161956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e5155-a895-4dc8-8c5b-bf860ac8a3cc admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:47.162238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e5155-a895-4dc8-8c5b-bf860ac8a3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:47.162567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834e5155-a895-4dc8-8c5b-bf860ac8a3cc admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:47.163237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e5155-a895-4dc8-8c5b-bf860ac8a3cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:47.165133 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834e5155-a895-4dc8-8c5b-bf860ac8a3cc admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:47.241662 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1247/2461] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:07:47 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:07:47.246317 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807a9b9-bd07-4edf-bef7-0d99e32de221 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:47.255818 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807a9b9-bd07-4edf-bef7-0d99e32de221 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:47.260422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629f3a2-8894-4075-a81c-fea7ec3b6f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:47.260998 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629f3a2-8894-4075-a81c-fea7ec3b6f64 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:47.261274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629f3a2-8894-4075-a81c-fea7ec3b6f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:47.262373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629f3a2-8894-4075-a81c-fea7ec3b6f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:47.263412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629f3a2-8894-4075-a81c-fea7ec3b6f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:47.265983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629f3a2-8894-4075-a81c-fea7ec3b6f64 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:47.271677 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1215/2462] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:07:47 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:07:47.277722 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e5155-a895-4dc8-8c5b-bf860ac8a3cc admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:47.281734 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1248/2463] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:07:47.285639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629f3a2-8894-4075-a81c-fea7ec3b6f64 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:47.295308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629f3a2-8894-4075-a81c-fea7ec3b6f64 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:47.299202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:47.340654 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834e5155-a895-4dc8-8c5b-bf860ac8a3cc admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:47.345432 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b89ff0-14b2-424c-82df-03bacf2ea7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:47.346041 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b89ff0-14b2-424c-82df-03bacf2ea7d9 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:47.346336 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b89ff0-14b2-424c-82df-03bacf2ea7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:47.346588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b89ff0-14b2-424c-82df-03bacf2ea7d9 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:47.347232 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b89ff0-14b2-424c-82df-03bacf2ea7d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:47.349153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b89ff0-14b2-424c-82df-03bacf2ea7d9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:47.353233 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d83f58-a8a1-4b5a-b701-95d20f269f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:47.353837 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d83f58-a8a1-4b5a-b701-95d20f269f0d admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:47.354114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d83f58-a8a1-4b5a-b701-95d20f269f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:47.354385 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d83f58-a8a1-4b5a-b701-95d20f269f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:47.355604 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d83f58-a8a1-4b5a-b701-95d20f269f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:47.357202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d83f58-a8a1-4b5a-b701-95d20f269f0d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:47.367367 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1249/2464] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:07:47.372037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d83f58-a8a1-4b5a-b701-95d20f269f0d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:47.382312 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d83f58-a8a1-4b5a-b701-95d20f269f0d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:47.387133 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d568b-e8e5-4f42-966f-d797f5e4744e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:47.387758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d568b-e8e5-4f42-966f-d797f5e4744e admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:47.388038 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d568b-e8e5-4f42-966f-d797f5e4744e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:47.388310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d568b-e8e5-4f42-966f-d797f5e4744e admin admin] PATH_INFO: `/v3/projects/6eee664f985046148019f1025bb5d924/users/f5d364f0c7654b38af60f5e77d54b914/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:47.403141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d568b-e8e5-4f42-966f-d797f5e4744e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eee664f985046148019f1025bb5d924, user_id=f5d364f0c7654b38af60f5e77d54b914, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:47.405949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d568b-e8e5-4f42-966f-d797f5e4744e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:47.443792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1216/2465] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:07:47 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:07:47.447798 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b89ff0-14b2-424c-82df-03bacf2ea7d9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:47.457399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b89ff0-14b2-424c-82df-03bacf2ea7d9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:47.462139 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f07819-1851-4889-af5a-1a8453a7b29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:47.462717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f07819-1851-4889-af5a-1a8453a7b29a admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:47.463032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f07819-1851-4889-af5a-1a8453a7b29a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:47.463310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f07819-1851-4889-af5a-1a8453a7b29a admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:47.463940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f07819-1851-4889-af5a-1a8453a7b29a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:47.465579 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f07819-1851-4889-af5a-1a8453a7b29a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:47.479342 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1217/2466] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:07:47.480598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f07819-1851-4889-af5a-1a8453a7b29a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:47.484717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1250/2467] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:07:47 2022] PUT /identity/v3/projects/6eee664f985046148019f1025bb5d924/users/f5d364f0c7654b38af60f5e77d54b914/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:07:47.496833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f07819-1851-4889-af5a-1a8453a7b29a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:47.505040 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52034caf-170c-4657-9dab-d456315c7256 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:47.505480 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52034caf-170c-4657-9dab-d456315c7256 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:47.505815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52034caf-170c-4657-9dab-d456315c7256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:47.534219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-52034caf-170c-4657-9dab-d456315c7256 None None] MFA Rules not processed for user `f5d364f0c7654b38af60f5e77d54b914`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:47.535458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-52034caf-170c-4657-9dab-d456315c7256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:47.563822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d59cac-b6d5-42d7-a651-c527a7395d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:47.564483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d59cac-b6d5-42d7-a651-c527a7395d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:47.564763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d59cac-b6d5-42d7-a651-c527a7395d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:47.565043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d59cac-b6d5-42d7-a651-c527a7395d7a admin admin] PATH_INFO: `/v3/projects/7e9d39a1dca64f6f851f1ad29a2cc952/users/25c907af9b9e4d968ed0127cc2a8cf4b/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:47.580526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d59cac-b6d5-42d7-a651-c527a7395d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9d39a1dca64f6f851f1ad29a2cc952, user_id=25c907af9b9e4d968ed0127cc2a8cf4b, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:47.583226 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d59cac-b6d5-42d7-a651-c527a7395d7a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:47.618607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1251/2468] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:07:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:07:47.625421 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52034caf-170c-4657-9dab-d456315c7256 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:47.640285 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52034caf-170c-4657-9dab-d456315c7256 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:47.649199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1218/2469] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:07:47 2022] PUT /identity/v3/projects/7e9d39a1dca64f6f851f1ad29a2cc952/users/25c907af9b9e4d968ed0127cc2a8cf4b/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:47.650216 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:47.653196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d59cac-b6d5-42d7-a651-c527a7395d7a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:47.703726 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fa8a2-a3f5-4af2-a9ce-ef9d0eec7d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:47.704509 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fa8a2-a3f5-4af2-a9ce-ef9d0eec7d16 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:47.704798 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fa8a2-a3f5-4af2-a9ce-ef9d0eec7d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:47.705074 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fa8a2-a3f5-4af2-a9ce-ef9d0eec7d16 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:47.706047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fa8a2-a3f5-4af2-a9ce-ef9d0eec7d16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:47.709083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fa8a2-a3f5-4af2-a9ce-ef9d0eec7d16 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:47.722342 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d59cac-b6d5-42d7-a651-c527a7395d7a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:47.727692 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35d9da-3125-45e8-8215-cf5346801f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:47.728321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35d9da-3125-45e8-8215-cf5346801f9c admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:47.728602 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35d9da-3125-45e8-8215-cf5346801f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:47.728952 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35d9da-3125-45e8-8215-cf5346801f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:47.729517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35d9da-3125-45e8-8215-cf5346801f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:47.731702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35d9da-3125-45e8-8215-cf5346801f9c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:47.742932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1219/2470] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:07:47.747736 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35d9da-3125-45e8-8215-cf5346801f9c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:47.757999 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35d9da-3125-45e8-8215-cf5346801f9c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:47.762921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f6fd0-1d9f-4180-868c-852dc1a25dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:47.763605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f6fd0-1d9f-4180-868c-852dc1a25dae admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:47.763881 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f6fd0-1d9f-4180-868c-852dc1a25dae admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:47.764238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f6fd0-1d9f-4180-868c-852dc1a25dae admin admin] PATH_INFO: `/v3/projects/7e9d39a1dca64f6f851f1ad29a2cc952/users/25c907af9b9e4d968ed0127cc2a8cf4b/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:47.766136 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f6fd0-1d9f-4180-868c-852dc1a25dae admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9d39a1dca64f6f851f1ad29a2cc952, user_id=25c907af9b9e4d968ed0127cc2a8cf4b, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:47.768815 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f6fd0-1d9f-4180-868c-852dc1a25dae admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:47.780790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1220/2471] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:07:47 2022] PUT /identity/v3/projects/7e9d39a1dca64f6f851f1ad29a2cc952/users/25c907af9b9e4d968ed0127cc2a8cf4b/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:07:47.795020 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbf4ee-d4d4-4099-8fa9-4bb040f6bff2 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:47.795299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbf4ee-d4d4-4099-8fa9-4bb040f6bff2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:47.795569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbf4ee-d4d4-4099-8fa9-4bb040f6bff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:47.805326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:47.805932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1252/2472] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:07:47 2022] POST /identity/v3/projects => generated 379 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:07:47.809605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fa8a2-a3f5-4af2-a9ce-ef9d0eec7d16 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:47.825903 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-31dbf4ee-d4d4-4099-8fa9-4bb040f6bff2 None None] MFA Rules not processed for user `25c907af9b9e4d968ed0127cc2a8cf4b`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:47.827360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-31dbf4ee-d4d4-4099-8fa9-4bb040f6bff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:47.876621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fa8a2-a3f5-4af2-a9ce-ef9d0eec7d16 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:47.881465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd6b30-d161-4453-958f-17d7ac9dd47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:47.882137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd6b30-d161-4453-958f-17d7ac9dd47e admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:47.882414 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd6b30-d161-4453-958f-17d7ac9dd47e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:47.882684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd6b30-d161-4453-958f-17d7ac9dd47e admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:47.883518 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd6b30-d161-4453-958f-17d7ac9dd47e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:47.885426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd6b30-d161-4453-958f-17d7ac9dd47e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:47.901819 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1221/2473] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:07:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:07:47.913776 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbf4ee-d4d4-4099-8fa9-4bb040f6bff2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:47.915206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-31dbf4ee-d4d4-4099-8fa9-4bb040f6bff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:47.974322 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbf4ee-d4d4-4099-8fa9-4bb040f6bff2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:47.977082 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda25c4-0cd7-4ef8-a9eb-31b3e75f90de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:47.977717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda25c4-0cd7-4ef8-a9eb-31b3e75f90de service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:47.978000 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda25c4-0cd7-4ef8-a9eb-31b3e75f90de service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:47.978275 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda25c4-0cd7-4ef8-a9eb-31b3e75f90de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:47.992026 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda25c4-0cd7-4ef8-a9eb-31b3e75f90de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:47.993340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda25c4-0cd7-4ef8-a9eb-31b3e75f90de service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:48.002788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1253/2474] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:07:47 2022] POST /identity/v3/users => generated 296 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:07:48.004463 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873b7a6-f4f0-47c7-8b0a-e6d4b21c31ed None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:48.004752 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873b7a6-f4f0-47c7-8b0a-e6d4b21c31ed None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:48.005031 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873b7a6-f4f0-47c7-8b0a-e6d4b21c31ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:48.007054 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1222/2475] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:07:48.007980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda25c4-0cd7-4ef8-a9eb-31b3e75f90de service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:48.018364 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda25c4-0cd7-4ef8-a9eb-31b3e75f90de service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:48.024653 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd577bbb-be66-499f-926b-d61ae2af7525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:48.025330 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd577bbb-be66-499f-926b-d61ae2af7525 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:48.025682 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd577bbb-be66-499f-926b-d61ae2af7525 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:48.026304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-0873b7a6-f4f0-47c7-8b0a-e6d4b21c31ed None None] MFA Rules not processed for user `b4cc11524fe542c5a5361b3630e6d3a6`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:48.026543 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd577bbb-be66-499f-926b-d61ae2af7525 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:48.028189 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd577bbb-be66-499f-926b-d61ae2af7525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:48.028308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0873b7a6-f4f0-47c7-8b0a-e6d4b21c31ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:48.032068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd577bbb-be66-499f-926b-d61ae2af7525 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:48.040397 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1223/2476] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:07:48.045199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd577bbb-be66-499f-926b-d61ae2af7525 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:48.055718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd577bbb-be66-499f-926b-d61ae2af7525 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:48.060575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06004bff-955a-431f-b043-f207b8666d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:48.061327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06004bff-955a-431f-b043-f207b8666d81 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:48.065721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06004bff-955a-431f-b043-f207b8666d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:48.065721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06004bff-955a-431f-b043-f207b8666d81 admin admin] PATH_INFO: `/v3/projects/e2d166aaffca4894878434172c3e4329/users/ec24a5cf9c774866b0f765e9df9ee8fd/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:48.077697 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06004bff-955a-431f-b043-f207b8666d81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d166aaffca4894878434172c3e4329, user_id=ec24a5cf9c774866b0f765e9df9ee8fd, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:48.080491 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06004bff-955a-431f-b043-f207b8666d81 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:48.085215 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1254/2477] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:07:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:07:48.161442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1224/2478] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:07:48 2022] PUT /identity/v3/projects/e2d166aaffca4894878434172c3e4329/users/ec24a5cf9c774866b0f765e9df9ee8fd/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:07:48.172056 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873b7a6-f4f0-47c7-8b0a-e6d4b21c31ed None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:48.183157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873b7a6-f4f0-47c7-8b0a-e6d4b21c31ed None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:48.213221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:48.249947 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d32ea-e9be-447e-8a03-920dc80789bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:48.250159 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06004bff-955a-431f-b043-f207b8666d81 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:48.250931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d32ea-e9be-447e-8a03-920dc80789bb admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:48.251315 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d32ea-e9be-447e-8a03-920dc80789bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:48.251658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d32ea-e9be-447e-8a03-920dc80789bb admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:48.252440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d32ea-e9be-447e-8a03-920dc80789bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:48.254297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d32ea-e9be-447e-8a03-920dc80789bb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:48.270071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1255/2479] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:07:48.271400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d32ea-e9be-447e-8a03-920dc80789bb admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:48.282332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d32ea-e9be-447e-8a03-920dc80789bb admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:48.287743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd171d9c-f98d-4693-b394-4d199fad983a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:48.288394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd171d9c-f98d-4693-b394-4d199fad983a admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:48.288671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd171d9c-f98d-4693-b394-4d199fad983a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:48.288944 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd171d9c-f98d-4693-b394-4d199fad983a admin admin] PATH_INFO: `/v3/projects/e2d166aaffca4894878434172c3e4329/users/ec24a5cf9c774866b0f765e9df9ee8fd/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:48.291566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd171d9c-f98d-4693-b394-4d199fad983a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2d166aaffca4894878434172c3e4329, user_id=ec24a5cf9c774866b0f765e9df9ee8fd, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:48.294638 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd171d9c-f98d-4693-b394-4d199fad983a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:48.312313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1256/2480] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:07:48 2022] PUT /identity/v3/projects/e2d166aaffca4894878434172c3e4329/users/ec24a5cf9c774866b0f765e9df9ee8fd/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:48.317419 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7332d0-7271-4b71-83ff-a0872c4a9549 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:48.317703 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7332d0-7271-4b71-83ff-a0872c4a9549 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:48.318001 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7332d0-7271-4b71-83ff-a0872c4a9549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:48.330845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06004bff-955a-431f-b043-f207b8666d81 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:48.341290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:48.351899 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-4b7332d0-7271-4b71-83ff-a0872c4a9549 None None] MFA Rules not processed for user `ec24a5cf9c774866b0f765e9df9ee8fd`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:48.353583 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4b7332d0-7271-4b71-83ff-a0872c4a9549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:48.386970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298ab58-6317-43fa-b21a-e8f615b99af8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:48.387807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298ab58-6317-43fa-b21a-e8f615b99af8 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:48.388188 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298ab58-6317-43fa-b21a-e8f615b99af8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:48.388568 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298ab58-6317-43fa-b21a-e8f615b99af8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:48.436913 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1257/2481] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:07:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:07:48.448287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7332d0-7271-4b71-83ff-a0872c4a9549 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:48.450547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4b7332d0-7271-4b71-83ff-a0872c4a9549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:48.461744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298ab58-6317-43fa-b21a-e8f615b99af8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:48.462643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7332d0-7271-4b71-83ff-a0872c4a9549 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:48.463356 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298ab58-6317-43fa-b21a-e8f615b99af8 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:48.467823 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695151a9-99ef-4428-bacf-03960b112422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:48.468458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695151a9-99ef-4428-bacf-03960b112422 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:48.468731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695151a9-99ef-4428-bacf-03960b112422 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:48.472564 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695151a9-99ef-4428-bacf-03960b112422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:48.479227 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1225/2482] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:07:48.486076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695151a9-99ef-4428-bacf-03960b112422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:48.490176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695151a9-99ef-4428-bacf-03960b112422 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:48.504751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1258/2483] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:07:48.746502 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298ab58-6317-43fa-b21a-e8f615b99af8 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:48.754937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298ab58-6317-43fa-b21a-e8f615b99af8 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:48.758513 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891131e-036e-40a7-89fc-de121ef763b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:48.759133 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891131e-036e-40a7-89fc-de121ef763b2 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:48.759407 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891131e-036e-40a7-89fc-de121ef763b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:48.759686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891131e-036e-40a7-89fc-de121ef763b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:48.819493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891131e-036e-40a7-89fc-de121ef763b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:48.820958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891131e-036e-40a7-89fc-de121ef763b2 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:48.831218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1226/2484] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:07:58.497773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=60,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:58.826995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=70 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:02.957762 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695151a9-99ef-4428-bacf-03960b112422 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:03.030697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695151a9-99ef-4428-bacf-03960b112422 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:03.042022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb94bae-14fb-4567-b40e-b8b2c10cb215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:03.042781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb94bae-14fb-4567-b40e-b8b2c10cb215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:03.043091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb94bae-14fb-4567-b40e-b8b2c10cb215 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:03.043375 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb94bae-14fb-4567-b40e-b8b2c10cb215 admin admin] PATH_INFO: `/v3/users/f5d364f0c7654b38af60f5e77d54b914` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:03.045450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb94bae-14fb-4567-b40e-b8b2c10cb215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5d364f0c7654b38af60f5e77d54b914)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:03.047996 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb94bae-14fb-4567-b40e-b8b2c10cb215 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:03.232670 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4fb94bae-14fb-4567-b40e-b8b2c10cb215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5d364f0c7654b38af60f5e77d54b914'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:03.281092 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891131e-036e-40a7-89fc-de121ef763b2 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:03.294045 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891131e-036e-40a7-89fc-de121ef763b2 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:03.299280 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90ce14-f310-4dda-9703-b1b394156328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:03.299946 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90ce14-f310-4dda-9703-b1b394156328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:03.300251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90ce14-f310-4dda-9703-b1b394156328 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:03.300808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90ce14-f310-4dda-9703-b1b394156328 admin admin] PATH_INFO: `/v3/users/566ff7aaedc1402e98ce303813003aa4` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:03.301944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90ce14-f310-4dda-9703-b1b394156328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=566ff7aaedc1402e98ce303813003aa4)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:03.303941 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90ce14-f310-4dda-9703-b1b394156328 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:03.317541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4fb94bae-14fb-4567-b40e-b8b2c10cb215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5d364f0c7654b38af60f5e77d54b914'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:03.320836 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4fb94bae-14fb-4567-b40e-b8b2c10cb215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5d364f0c7654b38af60f5e77d54b914'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:03.328981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4fb94bae-14fb-4567-b40e-b8b2c10cb215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5d364f0c7654b38af60f5e77d54b914'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:03.343450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4fb94bae-14fb-4567-b40e-b8b2c10cb215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5d364f0c7654b38af60f5e77d54b914'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:03.357995 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1259/2485] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:02 2022] DELETE /identity/v3/users/f5d364f0c7654b38af60f5e77d54b914 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:03.357995 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b860d0-15a5-4aaf-a928-a29b0393908a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:03.357995 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b860d0-15a5-4aaf-a928-a29b0393908a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:03.357995 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b860d0-15a5-4aaf-a928-a29b0393908a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:03.376136 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f7b860d0-15a5-4aaf-a928-a29b0393908a None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:03.377698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f7b860d0-15a5-4aaf-a928-a29b0393908a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:03.442162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1260/2486] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:03.452590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b860d0-15a5-4aaf-a928-a29b0393908a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:03.453990 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f7b860d0-15a5-4aaf-a928-a29b0393908a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:03.475830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:03.511881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b860d0-15a5-4aaf-a928-a29b0393908a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:03.516232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf20bc-b895-4012-a8ed-e1aab4e05fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:03.516833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf20bc-b895-4012-a8ed-e1aab4e05fdf service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:03.517108 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf20bc-b895-4012-a8ed-e1aab4e05fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:03.517373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf20bc-b895-4012-a8ed-e1aab4e05fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:03.527313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf20bc-b895-4012-a8ed-e1aab4e05fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:03.528925 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf20bc-b895-4012-a8ed-e1aab4e05fdf service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:03.530321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6c90ce14-f310-4dda-9703-b1b394156328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '566ff7aaedc1402e98ce303813003aa4'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:03.539459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1261/2487] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:08:03.605909 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6c90ce14-f310-4dda-9703-b1b394156328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '566ff7aaedc1402e98ce303813003aa4'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:03.621426 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6c90ce14-f310-4dda-9703-b1b394156328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '566ff7aaedc1402e98ce303813003aa4'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:03.622188 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6c90ce14-f310-4dda-9703-b1b394156328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '566ff7aaedc1402e98ce303813003aa4'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:03.630773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6c90ce14-f310-4dda-9703-b1b394156328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '566ff7aaedc1402e98ce303813003aa4'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:03.637073 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1227/2488] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:03 2022] DELETE /identity/v3/users/566ff7aaedc1402e98ce303813003aa4 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:03.642488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecde73c-1617-45fe-9a06-e25082563834 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:03.642783 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecde73c-1617-45fe-9a06-e25082563834 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:03.643102 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecde73c-1617-45fe-9a06-e25082563834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:03.663012 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1ecde73c-1617-45fe-9a06-e25082563834 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:03.669640 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1ecde73c-1617-45fe-9a06-e25082563834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:03.732864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1262/2489] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:03.743409 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90ce14-f310-4dda-9703-b1b394156328 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:03.744951 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6c90ce14-f310-4dda-9703-b1b394156328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:03.821460 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90ce14-f310-4dda-9703-b1b394156328 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:03.826850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce96af-7987-4625-8b1d-8f066acdfc49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:03.827859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce96af-7987-4625-8b1d-8f066acdfc49 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:03.827859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce96af-7987-4625-8b1d-8f066acdfc49 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:03.828287 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce96af-7987-4625-8b1d-8f066acdfc49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:03.839573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce96af-7987-4625-8b1d-8f066acdfc49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:03.841544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce96af-7987-4625-8b1d-8f066acdfc49 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:03.853109 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1228/2490] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:04.020771 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecde73c-1617-45fe-9a06-e25082563834 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:04.021298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1ecde73c-1617-45fe-9a06-e25082563834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:04.036511 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecde73c-1617-45fe-9a06-e25082563834 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:04.041902 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c46f96-bd82-441b-834a-c4b57df2d775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:04.050932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c46f96-bd82-441b-834a-c4b57df2d775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:04.050932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c46f96-bd82-441b-834a-c4b57df2d775 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:04.050932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c46f96-bd82-441b-834a-c4b57df2d775 admin admin] PATH_INFO: `/v3/projects/6eee664f985046148019f1025bb5d924` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:04.050932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c46f96-bd82-441b-834a-c4b57df2d775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eee664f985046148019f1025bb5d924)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:04.050932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c46f96-bd82-441b-834a-c4b57df2d775 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:04.177089 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-94c46f96-bd82-441b-834a-c4b57df2d775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eee664f985046148019f1025bb5d924', 'request_id': 'req-94c46f96-bd82-441b-834a-c4b57df2d775'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.292387 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-94c46f96-bd82-441b-834a-c4b57df2d775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eee664f985046148019f1025bb5d924', 'request_id': 'req-94c46f96-bd82-441b-834a-c4b57df2d775'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.294050 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce96af-7987-4625-8b1d-8f066acdfc49 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:04.296225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-95ce96af-7987-4625-8b1d-8f066acdfc49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:04.306476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-94c46f96-bd82-441b-834a-c4b57df2d775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eee664f985046148019f1025bb5d924', 'request_id': 'req-94c46f96-bd82-441b-834a-c4b57df2d775'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.313600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce96af-7987-4625-8b1d-8f066acdfc49 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:04.324643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-94c46f96-bd82-441b-834a-c4b57df2d775 admin admin] The token cache is being invalidate because project 6eee664f985046148019f1025bb5d924 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:04.324998 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-94c46f96-bd82-441b-834a-c4b57df2d775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.325600 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-94c46f96-bd82-441b-834a-c4b57df2d775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.330919 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1263/2491] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:04 2022] DELETE /identity/v3/projects/6eee664f985046148019f1025bb5d924 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:04.332138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c46f96-bd82-441b-834a-c4b57df2d775 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:04.335549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-94c46f96-bd82-441b-834a-c4b57df2d775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:04.387751 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dabe6-dda4-4c03-9b70-c5da73659aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:04.389266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dabe6-dda4-4c03-9b70-c5da73659aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:04.389618 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dabe6-dda4-4c03-9b70-c5da73659aef admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:04.389953 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dabe6-dda4-4c03-9b70-c5da73659aef admin admin] PATH_INFO: `/v3/projects/ddacd36c7ec04dd0bb485e4b32162a6f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:04.391207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94dabe6-dda4-4c03-9b70-c5da73659aef admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddacd36c7ec04dd0bb485e4b32162a6f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:04.391638 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:04.393404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94dabe6-dda4-4c03-9b70-c5da73659aef admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:04.409643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c46f96-bd82-441b-834a-c4b57df2d775 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:04.411489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:04.414613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223171e0-c732-4430-acec-4454803dcd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:04.415272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223171e0-c732-4430-acec-4454803dcd1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:04.416132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223171e0-c732-4430-acec-4454803dcd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:04.416132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223171e0-c732-4430-acec-4454803dcd1e admin admin] PATH_INFO: `/v3/users/ec24a5cf9c774866b0f765e9df9ee8fd` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:04.417007 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223171e0-c732-4430-acec-4454803dcd1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec24a5cf9c774866b0f765e9df9ee8fd)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:04.418957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223171e0-c732-4430-acec-4454803dcd1e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:04.499293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e94dabe6-dda4-4c03-9b70-c5da73659aef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddacd36c7ec04dd0bb485e4b32162a6f', 'request_id': 'req-e94dabe6-dda4-4c03-9b70-c5da73659aef'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.505712 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e94dabe6-dda4-4c03-9b70-c5da73659aef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddacd36c7ec04dd0bb485e4b32162a6f', 'request_id': 'req-e94dabe6-dda4-4c03-9b70-c5da73659aef'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.511926 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e94dabe6-dda4-4c03-9b70-c5da73659aef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddacd36c7ec04dd0bb485e4b32162a6f', 'request_id': 'req-e94dabe6-dda4-4c03-9b70-c5da73659aef'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.576984 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e94dabe6-dda4-4c03-9b70-c5da73659aef admin admin] The token cache is being invalidate because project ddacd36c7ec04dd0bb485e4b32162a6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:04.577304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e94dabe6-dda4-4c03-9b70-c5da73659aef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.577941 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e94dabe6-dda4-4c03-9b70-c5da73659aef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.579476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1229/2492] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:04 2022] DELETE /identity/v3/projects/ddacd36c7ec04dd0bb485e4b32162a6f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:04.583570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dabe6-dda4-4c03-9b70-c5da73659aef admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:04.585307 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e94dabe6-dda4-4c03-9b70-c5da73659aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:04.586279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-223171e0-c732-4430-acec-4454803dcd1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec24a5cf9c774866b0f765e9df9ee8fd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.649288 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dabe6-dda4-4c03-9b70-c5da73659aef admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:04.654777 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f68373-f259-4fba-b221-b312f2c61043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:04.655436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f68373-f259-4fba-b221-b312f2c61043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:04.662000 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f68373-f259-4fba-b221-b312f2c61043 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:04.662000 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f68373-f259-4fba-b221-b312f2c61043 admin admin] PATH_INFO: `/v3/users/25c907af9b9e4d968ed0127cc2a8cf4b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:04.662000 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f68373-f259-4fba-b221-b312f2c61043 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25c907af9b9e4d968ed0127cc2a8cf4b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:04.662000 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f68373-f259-4fba-b221-b312f2c61043 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:04.680468 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-223171e0-c732-4430-acec-4454803dcd1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec24a5cf9c774866b0f765e9df9ee8fd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.681126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-223171e0-c732-4430-acec-4454803dcd1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec24a5cf9c774866b0f765e9df9ee8fd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.689609 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-223171e0-c732-4430-acec-4454803dcd1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec24a5cf9c774866b0f765e9df9ee8fd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.701543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-223171e0-c732-4430-acec-4454803dcd1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec24a5cf9c774866b0f765e9df9ee8fd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.709052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1264/2493] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:04 2022] DELETE /identity/v3/users/ec24a5cf9c774866b0f765e9df9ee8fd => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:04.900224 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-16f68373-f259-4fba-b221-b312f2c61043 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25c907af9b9e4d968ed0127cc2a8cf4b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.954500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-16f68373-f259-4fba-b221-b312f2c61043 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25c907af9b9e4d968ed0127cc2a8cf4b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.973495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-16f68373-f259-4fba-b221-b312f2c61043 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25c907af9b9e4d968ed0127cc2a8cf4b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.974875 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-16f68373-f259-4fba-b221-b312f2c61043 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25c907af9b9e4d968ed0127cc2a8cf4b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.985685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-16f68373-f259-4fba-b221-b312f2c61043 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25c907af9b9e4d968ed0127cc2a8cf4b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:04.994558 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1230/2494] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:04 2022] DELETE /identity/v3/users/25c907af9b9e4d968ed0127cc2a8cf4b => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:05.188663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223171e0-c732-4430-acec-4454803dcd1e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:05.188663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-223171e0-c732-4430-acec-4454803dcd1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:05.255631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223171e0-c732-4430-acec-4454803dcd1e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:05.318585 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b045821-b1ee-4027-9ff8-78a75275652d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:05.319249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b045821-b1ee-4027-9ff8-78a75275652d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:05.319525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b045821-b1ee-4027-9ff8-78a75275652d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:05.319792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b045821-b1ee-4027-9ff8-78a75275652d admin admin] PATH_INFO: `/v3/projects/e2d166aaffca4894878434172c3e4329` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:05.320915 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b045821-b1ee-4027-9ff8-78a75275652d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2d166aaffca4894878434172c3e4329)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:05.322836 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b045821-b1ee-4027-9ff8-78a75275652d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:05.342690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:05.485220 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5b045821-b1ee-4027-9ff8-78a75275652d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2d166aaffca4894878434172c3e4329', 'request_id': 'req-5b045821-b1ee-4027-9ff8-78a75275652d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:05.488186 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f68373-f259-4fba-b221-b312f2c61043 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:05.491893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-16f68373-f259-4fba-b221-b312f2c61043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:05.506167 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f68373-f259-4fba-b221-b312f2c61043 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:05.530372 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:05.580681 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dc433a-94bd-4a82-bb16-25ff199f03dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:05.581393 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc433a-94bd-4a82-bb16-25ff199f03dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:05.581737 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc433a-94bd-4a82-bb16-25ff199f03dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:05.582013 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dc433a-94bd-4a82-bb16-25ff199f03dd admin admin] PATH_INFO: `/v3/projects/7e9d39a1dca64f6f851f1ad29a2cc952` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:05.583325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc433a-94bd-4a82-bb16-25ff199f03dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e9d39a1dca64f6f851f1ad29a2cc952)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:05.585244 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dc433a-94bd-4a82-bb16-25ff199f03dd admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:05.585890 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5b045821-b1ee-4027-9ff8-78a75275652d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2d166aaffca4894878434172c3e4329', 'request_id': 'req-5b045821-b1ee-4027-9ff8-78a75275652d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:05.593827 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5b045821-b1ee-4027-9ff8-78a75275652d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2d166aaffca4894878434172c3e4329', 'request_id': 'req-5b045821-b1ee-4027-9ff8-78a75275652d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:05.604612 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5b045821-b1ee-4027-9ff8-78a75275652d admin admin] The token cache is being invalidate because project e2d166aaffca4894878434172c3e4329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:05.604897 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5b045821-b1ee-4027-9ff8-78a75275652d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:05.605546 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5b045821-b1ee-4027-9ff8-78a75275652d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:05.607767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1265/2495] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:05 2022] DELETE /identity/v3/projects/e2d166aaffca4894878434172c3e4329 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:05.612382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bad0a-9316-4622-942b-57b6a81dc072 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:05.612664 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bad0a-9316-4622-942b-57b6a81dc072 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:05.612938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bad0a-9316-4622-942b-57b6a81dc072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:05.631974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-063bad0a-9316-4622-942b-57b6a81dc072 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:05.633175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-063bad0a-9316-4622-942b-57b6a81dc072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:05.641589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1266/2496] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:05.649214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e093c28-b0f3-423d-8bb2-8062b1e878ee None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:05.649496 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e093c28-b0f3-423d-8bb2-8062b1e878ee None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:05.649768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e093c28-b0f3-423d-8bb2-8062b1e878ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:05.670412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-4e093c28-b0f3-423d-8bb2-8062b1e878ee None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:05.671624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4e093c28-b0f3-423d-8bb2-8062b1e878ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:05.679488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1267/2497] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:05.683472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e093c28-b0f3-423d-8bb2-8062b1e878ee None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:05.693235 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e093c28-b0f3-423d-8bb2-8062b1e878ee None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:05.708685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c3dc433a-94bd-4a82-bb16-25ff199f03dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e9d39a1dca64f6f851f1ad29a2cc952', 'request_id': 'req-c3dc433a-94bd-4a82-bb16-25ff199f03dd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:05.714006 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c3dc433a-94bd-4a82-bb16-25ff199f03dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e9d39a1dca64f6f851f1ad29a2cc952', 'request_id': 'req-c3dc433a-94bd-4a82-bb16-25ff199f03dd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:05.721023 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c3dc433a-94bd-4a82-bb16-25ff199f03dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e9d39a1dca64f6f851f1ad29a2cc952', 'request_id': 'req-c3dc433a-94bd-4a82-bb16-25ff199f03dd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:05.752409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6300f-e8d8-45db-a9a5-b5dcaef756f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:05.753021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6300f-e8d8-45db-a9a5-b5dcaef756f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:05.753301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6300f-e8d8-45db-a9a5-b5dcaef756f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:05.753571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6300f-e8d8-45db-a9a5-b5dcaef756f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:05.754207 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6300f-e8d8-45db-a9a5-b5dcaef756f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:08:05.754508 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6300f-e8d8-45db-a9a5-b5dcaef756f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:05.756171 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6300f-e8d8-45db-a9a5-b5dcaef756f3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:05.774583 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1268/2498] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:08:05.776564 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6300f-e8d8-45db-a9a5-b5dcaef756f3 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:05.787070 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6300f-e8d8-45db-a9a5-b5dcaef756f3 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:05.791771 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004a341-9fa6-4ad2-bd49-939c8e6e62bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:05.792420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004a341-9fa6-4ad2-bd49-939c8e6e62bd admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:05.792635 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004a341-9fa6-4ad2-bd49-939c8e6e62bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:05.792902 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004a341-9fa6-4ad2-bd49-939c8e6e62bd admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:05.793524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004a341-9fa6-4ad2-bd49-939c8e6e62bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:05.796117 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004a341-9fa6-4ad2-bd49-939c8e6e62bd admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:05.798083 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c3dc433a-94bd-4a82-bb16-25ff199f03dd admin admin] The token cache is being invalidate because project 7e9d39a1dca64f6f851f1ad29a2cc952 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:05.800980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c3dc433a-94bd-4a82-bb16-25ff199f03dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:05.800980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c3dc433a-94bd-4a82-bb16-25ff199f03dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:05.801328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1231/2499] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:05 2022] DELETE /identity/v3/projects/7e9d39a1dca64f6f851f1ad29a2cc952 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:05.805578 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc8baf-9f7a-42f5-a134-0d8ed97a4da3 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:05.805859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc8baf-9f7a-42f5-a134-0d8ed97a4da3 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:05.806137 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc8baf-9f7a-42f5-a134-0d8ed97a4da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:05.826528 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-eebc8baf-9f7a-42f5-a134-0d8ed97a4da3 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:05.827720 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-eebc8baf-9f7a-42f5-a134-0d8ed97a4da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:05.835595 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1232/2500] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:05.845600 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d78fb-eb06-4a83-b8f6-670f6ad8ce23 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:05.845889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d78fb-eb06-4a83-b8f6-670f6ad8ce23 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:05.846162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d78fb-eb06-4a83-b8f6-670f6ad8ce23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:05.866060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-3f2d78fb-eb06-4a83-b8f6-670f6ad8ce23 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:05.867304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3f2d78fb-eb06-4a83-b8f6-670f6ad8ce23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:05.876941 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1233/2501] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:05.880512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1269/2502] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:05 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:05.882705 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d78fb-eb06-4a83-b8f6-670f6ad8ce23 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:05.884614 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004a341-9fa6-4ad2-bd49-939c8e6e62bd admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:05.886782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e004a341-9fa6-4ad2-bd49-939c8e6e62bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:05.894332 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d78fb-eb06-4a83-b8f6-670f6ad8ce23 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:05.957239 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004a341-9fa6-4ad2-bd49-939c8e6e62bd admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:05.959930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937c1ca-b9b5-471f-8c62-da60bff396f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:05.960561 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937c1ca-b9b5-471f-8c62-da60bff396f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:05.960860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937c1ca-b9b5-471f-8c62-da60bff396f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:05.961135 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937c1ca-b9b5-471f-8c62-da60bff396f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:05.961785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937c1ca-b9b5-471f-8c62-da60bff396f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:08:05.961973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb29500-8a2a-4f95-98fd-4d9dae293bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:05.962089 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937c1ca-b9b5-471f-8c62-da60bff396f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:05.962570 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb29500-8a2a-4f95-98fd-4d9dae293bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:05.962847 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb29500-8a2a-4f95-98fd-4d9dae293bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:05.963158 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb29500-8a2a-4f95-98fd-4d9dae293bb6 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:05.963750 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3937c1ca-b9b5-471f-8c62-da60bff396f3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:05.963851 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb29500-8a2a-4f95-98fd-4d9dae293bb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:05.965878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb29500-8a2a-4f95-98fd-4d9dae293bb6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:05.975768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:05.976726 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1234/2503] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:08:05.984626 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937c1ca-b9b5-471f-8c62-da60bff396f3 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:05.992377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937c1ca-b9b5-471f-8c62-da60bff396f3 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:05.997482 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad960de-46ce-4e47-8762-1bce6b8efaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:05.998060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad960de-46ce-4e47-8762-1bce6b8efaed admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:05.998349 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad960de-46ce-4e47-8762-1bce6b8efaed admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:05.998667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad960de-46ce-4e47-8762-1bce6b8efaed admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:05.999418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad960de-46ce-4e47-8762-1bce6b8efaed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:06.001341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad960de-46ce-4e47-8762-1bce6b8efaed admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:06.071683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1270/2504] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:05 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:06.076240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb29500-8a2a-4f95-98fd-4d9dae293bb6 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:06.086035 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb29500-8a2a-4f95-98fd-4d9dae293bb6 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:06.091414 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a8eda-f05c-44ff-a19f-09ef2c129181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:06.092557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a8eda-f05c-44ff-a19f-09ef2c129181 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:06.092849 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a8eda-f05c-44ff-a19f-09ef2c129181 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:06.093121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a8eda-f05c-44ff-a19f-09ef2c129181 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:06.093761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a8eda-f05c-44ff-a19f-09ef2c129181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:06.095459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a8eda-f05c-44ff-a19f-09ef2c129181 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:06.096204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1235/2505] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:05 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:06.100491 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad960de-46ce-4e47-8762-1bce6b8efaed admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:06.106021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1271/2506] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:06.109918 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a8eda-f05c-44ff-a19f-09ef2c129181 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:06.119739 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a8eda-f05c-44ff-a19f-09ef2c129181 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:06.163673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad960de-46ce-4e47-8762-1bce6b8efaed admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:06.169832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c9433-4019-4e58-99dc-0a2aa1481980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:06.170451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c9433-4019-4e58-99dc-0a2aa1481980 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:06.170728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c9433-4019-4e58-99dc-0a2aa1481980 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:06.171045 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c9433-4019-4e58-99dc-0a2aa1481980 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:06.171662 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c9433-4019-4e58-99dc-0a2aa1481980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:06.173858 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c9433-4019-4e58-99dc-0a2aa1481980 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:06.176635 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b133d-e130-4696-b59b-c8742dff5357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:06.177322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b133d-e130-4696-b59b-c8742dff5357 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:06.177603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b133d-e130-4696-b59b-c8742dff5357 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:06.177876 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b133d-e130-4696-b59b-c8742dff5357 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:06.178510 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b133d-e130-4696-b59b-c8742dff5357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:06.180125 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b133d-e130-4696-b59b-c8742dff5357 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:06.190380 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1272/2507] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:06.194662 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b133d-e130-4696-b59b-c8742dff5357 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:06.205023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b133d-e130-4696-b59b-c8742dff5357 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:06.209827 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39222bac-33c3-4f68-bc11-d6c31bffeddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:06.210413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39222bac-33c3-4f68-bc11-d6c31bffeddd admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:06.210696 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39222bac-33c3-4f68-bc11-d6c31bffeddd admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:06.211008 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39222bac-33c3-4f68-bc11-d6c31bffeddd admin admin] PATH_INFO: `/v3/projects/174225c7cc2143e8af4c67b11e413f5b/users/06ee62074dc64ff6b46a81542a1d28be/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:06.214376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:06.226356 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39222bac-33c3-4f68-bc11-d6c31bffeddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=174225c7cc2143e8af4c67b11e413f5b, user_id=06ee62074dc64ff6b46a81542a1d28be, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:06.229071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39222bac-33c3-4f68-bc11-d6c31bffeddd admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:06.266533 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1236/2508] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:06 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:06.270747 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c9433-4019-4e58-99dc-0a2aa1481980 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:06.281210 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c9433-4019-4e58-99dc-0a2aa1481980 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:06.285783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e48b6-d218-4888-97ca-be6f458b1911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:06.286351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e48b6-d218-4888-97ca-be6f458b1911 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:06.288036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e48b6-d218-4888-97ca-be6f458b1911 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:06.288036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e48b6-d218-4888-97ca-be6f458b1911 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:06.288036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e48b6-d218-4888-97ca-be6f458b1911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:06.289247 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e48b6-d218-4888-97ca-be6f458b1911 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:06.299585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1237/2509] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:06.303340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e48b6-d218-4888-97ca-be6f458b1911 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:06.317925 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e48b6-d218-4888-97ca-be6f458b1911 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:06.318704 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a78ee-f6b5-485d-a28f-03eed8a2d1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:06.319371 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a78ee-f6b5-485d-a28f-03eed8a2d1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:06.319654 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a78ee-f6b5-485d-a28f-03eed8a2d1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:06.319932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a78ee-f6b5-485d-a28f-03eed8a2d1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:06.320563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a78ee-f6b5-485d-a28f-03eed8a2d1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:06.322135 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a78ee-f6b5-485d-a28f-03eed8a2d1a5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:06.333318 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1238/2510] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:06.337317 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a78ee-f6b5-485d-a28f-03eed8a2d1a5 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:06.347813 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a78ee-f6b5-485d-a28f-03eed8a2d1a5 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:06.351539 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1273/2511] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:06 2022] PUT /identity/v3/projects/174225c7cc2143e8af4c67b11e413f5b/users/06ee62074dc64ff6b46a81542a1d28be/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:06.363277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86e8bd-fac4-4594-acf5-ffc8b51db39c None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:06.363559 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86e8bd-fac4-4594-acf5-ffc8b51db39c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:06.363829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86e8bd-fac4-4594-acf5-ffc8b51db39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:06.392762 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3f86e8bd-fac4-4594-acf5-ffc8b51db39c None None] MFA Rules not processed for user `06ee62074dc64ff6b46a81542a1d28be`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:06.394008 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3f86e8bd-fac4-4594-acf5-ffc8b51db39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:06.406689 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea26459-1d52-4014-8324-987ab05eae9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:06.407405 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea26459-1d52-4014-8324-987ab05eae9a admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:06.407688 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea26459-1d52-4014-8324-987ab05eae9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:06.407956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea26459-1d52-4014-8324-987ab05eae9a admin admin] PATH_INFO: `/v3/projects/58e42f40ea83476cafb97c10bb4f4996/users/90bccc082ccc4358800399bea08dd3cd/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:06.423644 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea26459-1d52-4014-8324-987ab05eae9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=58e42f40ea83476cafb97c10bb4f4996, user_id=90bccc082ccc4358800399bea08dd3cd, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:06.426363 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea26459-1d52-4014-8324-987ab05eae9a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:06.460301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1274/2512] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:06.464761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86e8bd-fac4-4594-acf5-ffc8b51db39c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:06.474658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86e8bd-fac4-4594-acf5-ffc8b51db39c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:06.479186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4f20c-fab6-485b-bda4-611c59f27c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:06.479789 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4f20c-fab6-485b-bda4-611c59f27c27 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:06.480054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4f20c-fab6-485b-bda4-611c59f27c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:06.480329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4f20c-fab6-485b-bda4-611c59f27c27 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:06.480956 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4f20c-fab6-485b-bda4-611c59f27c27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:06.482908 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4f20c-fab6-485b-bda4-611c59f27c27 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:06.497419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1239/2513] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:06 2022] PUT /identity/v3/projects/58e42f40ea83476cafb97c10bb4f4996/users/90bccc082ccc4358800399bea08dd3cd/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:06.509308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b1be2-4a64-45c6-9036-c4d44b9840d9 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:06.509595 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b1be2-4a64-45c6-9036-c4d44b9840d9 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:06.509902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b1be2-4a64-45c6-9036-c4d44b9840d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:06.538384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-412b1be2-4a64-45c6-9036-c4d44b9840d9 None None] MFA Rules not processed for user `90bccc082ccc4358800399bea08dd3cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:06.539609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-412b1be2-4a64-45c6-9036-c4d44b9840d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:06.573842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1275/2514] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:06 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:06.578443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4f20c-fab6-485b-bda4-611c59f27c27 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:06.601053 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:06.611738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1240/2515] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:06.616236 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b1be2-4a64-45c6-9036-c4d44b9840d9 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:06.630596 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:06.651533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4f20c-fab6-485b-bda4-611c59f27c27 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:06.657117 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa329116-83e3-43af-b164-2097af15326a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:06.658328 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa329116-83e3-43af-b164-2097af15326a admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:06.658456 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa329116-83e3-43af-b164-2097af15326a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:06.658784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa329116-83e3-43af-b164-2097af15326a admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:06.660009 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa329116-83e3-43af-b164-2097af15326a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:06.664689 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa329116-83e3-43af-b164-2097af15326a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:06.688400 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b1be2-4a64-45c6-9036-c4d44b9840d9 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:06.693454 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbe13d-266d-4d2f-9e0c-974a5d73ccb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:06.694067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbe13d-266d-4d2f-9e0c-974a5d73ccb3 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:06.694349 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbe13d-266d-4d2f-9e0c-974a5d73ccb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:06.694651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbe13d-266d-4d2f-9e0c-974a5d73ccb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:06.695330 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbe13d-266d-4d2f-9e0c-974a5d73ccb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:06.697432 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbe13d-266d-4d2f-9e0c-974a5d73ccb3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:06.738282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1276/2516] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:06 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:06.742643 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa329116-83e3-43af-b164-2097af15326a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:06.754390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa329116-83e3-43af-b164-2097af15326a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:06.759220 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4454446-e641-4ac1-9e02-90d13ad98077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:06.759822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4454446-e641-4ac1-9e02-90d13ad98077 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:06.760102 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4454446-e641-4ac1-9e02-90d13ad98077 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:06.760379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4454446-e641-4ac1-9e02-90d13ad98077 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:06.761020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4454446-e641-4ac1-9e02-90d13ad98077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:06.762608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4454446-e641-4ac1-9e02-90d13ad98077 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:06.777582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1277/2517] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:06.778693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4454446-e641-4ac1-9e02-90d13ad98077 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:06.793000 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1241/2518] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:06 2022] POST /identity/v3/projects => generated 381 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:06.793593 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4454446-e641-4ac1-9e02-90d13ad98077 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:06.797573 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbe13d-266d-4d2f-9e0c-974a5d73ccb3 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:06.853854 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6c31e-904c-4044-8bea-63b8d057929a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:06.854533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6c31e-904c-4044-8bea-63b8d057929a admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:06.854946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6c31e-904c-4044-8bea-63b8d057929a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:06.855224 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6c31e-904c-4044-8bea-63b8d057929a admin admin] PATH_INFO: `/v3/projects/b70cf49189954c638b9e19e3f06aa063/users/0d883c83c7c44dc48b0d588fe9bcde65/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:06.862659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbe13d-266d-4d2f-9e0c-974a5d73ccb3 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:06.868440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec5a3a-9fdb-45ad-9bc0-061eb6df4d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:06.869060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec5a3a-9fdb-45ad-9bc0-061eb6df4d5a admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:06.869365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec5a3a-9fdb-45ad-9bc0-061eb6df4d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:06.869653 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec5a3a-9fdb-45ad-9bc0-061eb6df4d5a admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:06.870321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec5a3a-9fdb-45ad-9bc0-061eb6df4d5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:06.870829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6c31e-904c-4044-8bea-63b8d057929a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b70cf49189954c638b9e19e3f06aa063, user_id=0d883c83c7c44dc48b0d588fe9bcde65, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:06.872278 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec5a3a-9fdb-45ad-9bc0-061eb6df4d5a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:06.873567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6c31e-904c-4044-8bea-63b8d057929a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:06.952125 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1278/2519] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:06 2022] PUT /identity/v3/projects/b70cf49189954c638b9e19e3f06aa063/users/0d883c83c7c44dc48b0d588fe9bcde65/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:06.956134 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6c31e-904c-4044-8bea-63b8d057929a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:06.963648 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1242/2520] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:06 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:06.968448 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec5a3a-9fdb-45ad-9bc0-061eb6df4d5a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:06.978351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec5a3a-9fdb-45ad-9bc0-061eb6df4d5a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:07.022205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6c31e-904c-4044-8bea-63b8d057929a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:07.028139 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c289f-7d1b-411e-aa33-70c0fd82f46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:07.028907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c289f-7d1b-411e-aa33-70c0fd82f46f admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:07.029240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c289f-7d1b-411e-aa33-70c0fd82f46f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:07.029589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c289f-7d1b-411e-aa33-70c0fd82f46f admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:07.030506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c289f-7d1b-411e-aa33-70c0fd82f46f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:07.032241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c289f-7d1b-411e-aa33-70c0fd82f46f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:07.041470 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9acba8-6677-448e-ab5f-b58c1215da11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:07.042073 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9acba8-6677-448e-ab5f-b58c1215da11 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:07.042351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9acba8-6677-448e-ab5f-b58c1215da11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:07.042622 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9acba8-6677-448e-ab5f-b58c1215da11 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:07.042727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1279/2521] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:07.043289 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9acba8-6677-448e-ab5f-b58c1215da11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:07.045467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9acba8-6677-448e-ab5f-b58c1215da11 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:07.047670 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c289f-7d1b-411e-aa33-70c0fd82f46f admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:07.058874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1243/2522] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:07.059088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c289f-7d1b-411e-aa33-70c0fd82f46f admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:07.059852 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9acba8-6677-448e-ab5f-b58c1215da11 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:07.063572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a02f9c-c657-4339-9099-7309729da6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:07.064226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a02f9c-c657-4339-9099-7309729da6d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:07.064508 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a02f9c-c657-4339-9099-7309729da6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:07.064788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a02f9c-c657-4339-9099-7309729da6d2 admin admin] PATH_INFO: `/v3/projects/b70cf49189954c638b9e19e3f06aa063/users/0d883c83c7c44dc48b0d588fe9bcde65/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:07.066749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a02f9c-c657-4339-9099-7309729da6d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b70cf49189954c638b9e19e3f06aa063, user_id=0d883c83c7c44dc48b0d588fe9bcde65, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:07.067272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:07.069605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a02f9c-c657-4339-9099-7309729da6d2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:07.071335 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9acba8-6677-448e-ab5f-b58c1215da11 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:07.076098 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67fecd2-cb10-4361-8c4a-305cd548eb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:07.089321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fecd2-cb10-4361-8c4a-305cd548eb08 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:07.089321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fecd2-cb10-4361-8c4a-305cd548eb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:07.089321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67fecd2-cb10-4361-8c4a-305cd548eb08 admin admin] PATH_INFO: `/v3/projects/78e9dacfab33473fac2c930c13dfd593/users/4e82ff8c72f24d11b542cb43b1ade2ab/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:07.089618 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1280/2523] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:07 2022] PUT /identity/v3/projects/b70cf49189954c638b9e19e3f06aa063/users/0d883c83c7c44dc48b0d588fe9bcde65/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:07.091171 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7f034-df45-41f1-93e1-f9a9bd6bb748 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:07.091612 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7f034-df45-41f1-93e1-f9a9bd6bb748 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:07.091894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7f034-df45-41f1-93e1-f9a9bd6bb748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:07.092664 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67fecd2-cb10-4361-8c4a-305cd548eb08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e9dacfab33473fac2c930c13dfd593, user_id=4e82ff8c72f24d11b542cb43b1ade2ab, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:07.095360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67fecd2-cb10-4361-8c4a-305cd548eb08 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:07.101051 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:07.120637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a4c7f034-df45-41f1-93e1-f9a9bd6bb748 None None] MFA Rules not processed for user `0d883c83c7c44dc48b0d588fe9bcde65`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:07.121903 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a4c7f034-df45-41f1-93e1-f9a9bd6bb748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:07.151972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1244/2524] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:07 2022] PUT /identity/v3/projects/78e9dacfab33473fac2c930c13dfd593/users/4e82ff8c72f24d11b542cb43b1ade2ab/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:07.156210 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67fecd2-cb10-4361-8c4a-305cd548eb08 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:07.194046 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1281/2525] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:07.206952 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7f034-df45-41f1-93e1-f9a9bd6bb748 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:07.208355 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a4c7f034-df45-41f1-93e1-f9a9bd6bb748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:07.236385 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67fecd2-cb10-4361-8c4a-305cd548eb08 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:07.241630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a5349-c540-4a5d-bb37-f12542c08347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:07.242226 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a5349-c540-4a5d-bb37-f12542c08347 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:07.242509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a5349-c540-4a5d-bb37-f12542c08347 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:07.242783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a5349-c540-4a5d-bb37-f12542c08347 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:07.243459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a5349-c540-4a5d-bb37-f12542c08347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:07.245151 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a5349-c540-4a5d-bb37-f12542c08347 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:07.255446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1245/2526] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:07.259476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a5349-c540-4a5d-bb37-f12542c08347 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:07.269651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a5349-c540-4a5d-bb37-f12542c08347 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:07.275554 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4a7cf-db12-4ed9-895d-59705df003eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:07.276201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4a7cf-db12-4ed9-895d-59705df003eb admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:07.276478 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4a7cf-db12-4ed9-895d-59705df003eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:07.276756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7f034-df45-41f1-93e1-f9a9bd6bb748 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:07.276883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4a7cf-db12-4ed9-895d-59705df003eb admin admin] PATH_INFO: `/v3/projects/78e9dacfab33473fac2c930c13dfd593/users/4e82ff8c72f24d11b542cb43b1ade2ab/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:07.279443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4a7cf-db12-4ed9-895d-59705df003eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e9dacfab33473fac2c930c13dfd593, user_id=4e82ff8c72f24d11b542cb43b1ade2ab, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:07.282103 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4a7cf-db12-4ed9-895d-59705df003eb admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:07.293052 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1246/2527] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:07 2022] PUT /identity/v3/projects/78e9dacfab33473fac2c930c13dfd593/users/4e82ff8c72f24d11b542cb43b1ade2ab/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:07.304069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5178e9b-8390-416e-8b2c-b5936ebd3458 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:07.304387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5178e9b-8390-416e-8b2c-b5936ebd3458 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:07.304625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5178e9b-8390-416e-8b2c-b5936ebd3458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:07.330780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f31437-b815-4503-944b-4c8e7acd601c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:07.331437 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f31437-b815-4503-944b-4c8e7acd601c service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:07.331719 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f31437-b815-4503-944b-4c8e7acd601c service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:07.332023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f31437-b815-4503-944b-4c8e7acd601c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:07.333346 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c5178e9b-8390-416e-8b2c-b5936ebd3458 None None] MFA Rules not processed for user `4e82ff8c72f24d11b542cb43b1ade2ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:07.334557 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c5178e9b-8390-416e-8b2c-b5936ebd3458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:07.400554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:07.402989 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f31437-b815-4503-944b-4c8e7acd601c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:07.404758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f31437-b815-4503-944b-4c8e7acd601c service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:07.413289 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1247/2528] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:07.415834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1282/2529] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:07.427111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5178e9b-8390-416e-8b2c-b5936ebd3458 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:07.429551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c5178e9b-8390-416e-8b2c-b5936ebd3458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:07.491702 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f31437-b815-4503-944b-4c8e7acd601c service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:07.498703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:07.503013 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f31437-b815-4503-944b-4c8e7acd601c service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:07.503297 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5178e9b-8390-416e-8b2c-b5936ebd3458 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:07.507176 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5970aaa-d31b-4beb-bb00-35eaa634eec7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:07.507783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5970aaa-d31b-4beb-bb00-35eaa634eec7 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:07.508056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5970aaa-d31b-4beb-bb00-35eaa634eec7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:07.508332 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5970aaa-d31b-4beb-bb00-35eaa634eec7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:07.518107 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5970aaa-d31b-4beb-bb00-35eaa634eec7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:07.519575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5970aaa-d31b-4beb-bb00-35eaa634eec7 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:07.531449 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1248/2530] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:07.565408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cf8b9-6aaf-4024-a868-e3f38de00a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:07.566180 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cf8b9-6aaf-4024-a868-e3f38de00a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:07.566458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cf8b9-6aaf-4024-a868-e3f38de00a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:07.566730 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cf8b9-6aaf-4024-a868-e3f38de00a96 admin admin] PATH_INFO: `/v3/users/06ee62074dc64ff6b46a81542a1d28be` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:07.567898 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25cf8b9-6aaf-4024-a868-e3f38de00a96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06ee62074dc64ff6b46a81542a1d28be)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:07.569851 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25cf8b9-6aaf-4024-a868-e3f38de00a96 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:07.811657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a25cf8b9-6aaf-4024-a868-e3f38de00a96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06ee62074dc64ff6b46a81542a1d28be'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:07.886024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a25cf8b9-6aaf-4024-a868-e3f38de00a96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06ee62074dc64ff6b46a81542a1d28be'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:07.886757 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a25cf8b9-6aaf-4024-a868-e3f38de00a96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06ee62074dc64ff6b46a81542a1d28be'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:07.896297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a25cf8b9-6aaf-4024-a868-e3f38de00a96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06ee62074dc64ff6b46a81542a1d28be'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:07.912839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a25cf8b9-6aaf-4024-a868-e3f38de00a96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06ee62074dc64ff6b46a81542a1d28be'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:07.920080 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1283/2531] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:07 2022] DELETE /identity/v3/users/06ee62074dc64ff6b46a81542a1d28be => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:07.924385 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d0ef8-5484-4322-90fc-b608f18ee076 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:07.924666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d0ef8-5484-4322-90fc-b608f18ee076 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:07.924973 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d0ef8-5484-4322-90fc-b608f18ee076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:07.946038 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-910d0ef8-5484-4322-90fc-b608f18ee076 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:07.947290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-910d0ef8-5484-4322-90fc-b608f18ee076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:08.009042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1249/2532] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:08.020062 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cf8b9-6aaf-4024-a868-e3f38de00a96 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:08.021467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a25cf8b9-6aaf-4024-a868-e3f38de00a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:08.083992 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cf8b9-6aaf-4024-a868-e3f38de00a96 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:08.087579 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b026781-fd58-4288-af73-9c0ee6069147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:08.088212 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b026781-fd58-4288-af73-9c0ee6069147 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:08.088505 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b026781-fd58-4288-af73-9c0ee6069147 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:08.088763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b026781-fd58-4288-af73-9c0ee6069147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:08.098073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b026781-fd58-4288-af73-9c0ee6069147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:08.099663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b026781-fd58-4288-af73-9c0ee6069147 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:08.109295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1284/2533] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:08.549112 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d0ef8-5484-4322-90fc-b608f18ee076 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:08.549112 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-910d0ef8-5484-4322-90fc-b608f18ee076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:08.560861 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d0ef8-5484-4322-90fc-b608f18ee076 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:08.565559 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66942e09-8313-4619-abfb-43e57b96a57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:08.566192 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66942e09-8313-4619-abfb-43e57b96a57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:08.566467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66942e09-8313-4619-abfb-43e57b96a57f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:08.566734 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66942e09-8313-4619-abfb-43e57b96a57f admin admin] PATH_INFO: `/v3/projects/174225c7cc2143e8af4c67b11e413f5b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:08.568150 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66942e09-8313-4619-abfb-43e57b96a57f admin admin] RBAC: Authorizing `identity:delete_project(project_id=174225c7cc2143e8af4c67b11e413f5b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:08.570067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66942e09-8313-4619-abfb-43e57b96a57f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:08.635616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b026781-fd58-4288-af73-9c0ee6069147 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:08.637039 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0b026781-fd58-4288-af73-9c0ee6069147 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:08.658752 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:08.687974 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-66942e09-8313-4619-abfb-43e57b96a57f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '174225c7cc2143e8af4c67b11e413f5b', 'request_id': 'req-66942e09-8313-4619-abfb-43e57b96a57f'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:08.694122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-66942e09-8313-4619-abfb-43e57b96a57f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '174225c7cc2143e8af4c67b11e413f5b', 'request_id': 'req-66942e09-8313-4619-abfb-43e57b96a57f'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:08.700567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-66942e09-8313-4619-abfb-43e57b96a57f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '174225c7cc2143e8af4c67b11e413f5b', 'request_id': 'req-66942e09-8313-4619-abfb-43e57b96a57f'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:08.707713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b026781-fd58-4288-af73-9c0ee6069147 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:08.837057 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-66942e09-8313-4619-abfb-43e57b96a57f admin admin] The token cache is being invalidate because project 174225c7cc2143e8af4c67b11e413f5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:08.837653 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-66942e09-8313-4619-abfb-43e57b96a57f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:08.838350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-66942e09-8313-4619-abfb-43e57b96a57f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:08.840096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1250/2534] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:08 2022] DELETE /identity/v3/projects/174225c7cc2143e8af4c67b11e413f5b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:08.844892 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66942e09-8313-4619-abfb-43e57b96a57f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:08.846363 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-66942e09-8313-4619-abfb-43e57b96a57f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:08.902809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2808e1c-f8bd-412c-a4ed-a288e6f2c9fc tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e82ff8c72f24d11b542cb43b1ade2ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78e9dacfab33473fac2c930c13dfd593', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:08.903543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2808e1c-f8bd-412c-a4ed-a288e6f2c9fc tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:08.904481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2808e1c-f8bd-412c-a4ed-a288e6f2c9fc tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:08.904780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2808e1c-f8bd-412c-a4ed-a288e6f2c9fc tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:08.906415 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2808e1c-f8bd-412c-a4ed-a288e6f2c9fc tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:08:08.906785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2808e1c-f8bd-412c-a4ed-a288e6f2c9fc tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:08.908923 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2808e1c-f8bd-412c-a4ed-a288e6f2c9fc tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:08.924360 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1285/2535] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:08:08.927572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2808e1c-f8bd-412c-a4ed-a288e6f2c9fc tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:08.927706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66942e09-8313-4619-abfb-43e57b96a57f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:08.929337 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d2808e1c-f8bd-412c-a4ed-a288e6f2c9fc tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:08.933062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1379186-92b2-4900-b6ed-997de18613c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:08.933833 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1379186-92b2-4900-b6ed-997de18613c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:08.934211 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1379186-92b2-4900-b6ed-997de18613c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:08.934584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1379186-92b2-4900-b6ed-997de18613c5 admin admin] PATH_INFO: `/v3/users/0d883c83c7c44dc48b0d588fe9bcde65` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:08.935883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1379186-92b2-4900-b6ed-997de18613c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d883c83c7c44dc48b0d588fe9bcde65)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:08.938267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1379186-92b2-4900-b6ed-997de18613c5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:08.943265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2808e1c-f8bd-412c-a4ed-a288e6f2c9fc tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:08.949070 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5bdec-39be-4509-8806-c3ad73e63270 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e82ff8c72f24d11b542cb43b1ade2ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78e9dacfab33473fac2c930c13dfd593', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:08.949784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5bdec-39be-4509-8806-c3ad73e63270 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:08.950184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5bdec-39be-4509-8806-c3ad73e63270 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:08.950535 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5bdec-39be-4509-8806-c3ad73e63270 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:08.951332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5bdec-39be-4509-8806-c3ad73e63270 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:08.953914 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5bdec-39be-4509-8806-c3ad73e63270 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:08.957857 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:09.079849 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1286/2536] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:08 2022] POST /identity/v3/projects => generated 355 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:09.083849 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5bdec-39be-4509-8806-c3ad73e63270 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:09.103542 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c1379186-92b2-4900-b6ed-997de18613c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d883c83c7c44dc48b0d588fe9bcde65'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:09.145605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5bdec-39be-4509-8806-c3ad73e63270 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:09.150632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6817c3-e03e-43a0-80ba-e4593808f7b5 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e82ff8c72f24d11b542cb43b1ade2ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78e9dacfab33473fac2c930c13dfd593', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:09.151293 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6817c3-e03e-43a0-80ba-e4593808f7b5 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:09.151647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6817c3-e03e-43a0-80ba-e4593808f7b5 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:09.151847 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6817c3-e03e-43a0-80ba-e4593808f7b5 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:09.152485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6817c3-e03e-43a0-80ba-e4593808f7b5 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:08:09.152791 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6817c3-e03e-43a0-80ba-e4593808f7b5 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:09.155154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6817c3-e03e-43a0-80ba-e4593808f7b5 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:09.167552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1287/2537] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:08:09.176997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c1379186-92b2-4900-b6ed-997de18613c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d883c83c7c44dc48b0d588fe9bcde65'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:09.191649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c1379186-92b2-4900-b6ed-997de18613c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d883c83c7c44dc48b0d588fe9bcde65'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:09.192392 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c1379186-92b2-4900-b6ed-997de18613c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d883c83c7c44dc48b0d588fe9bcde65'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:09.209150 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c1379186-92b2-4900-b6ed-997de18613c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d883c83c7c44dc48b0d588fe9bcde65'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:09.209338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6817c3-e03e-43a0-80ba-e4593808f7b5 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:09.209338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ed6817c3-e03e-43a0-80ba-e4593808f7b5 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:09.209531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1251/2538] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:08 2022] DELETE /identity/v3/users/0d883c83c7c44dc48b0d588fe9bcde65 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:09.222976 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6817c3-e03e-43a0-80ba-e4593808f7b5 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:09.253007 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:09.293096 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40b6e23-93ef-4256-944f-88e9bb11ae57 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e82ff8c72f24d11b542cb43b1ade2ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '78e9dacfab33473fac2c930c13dfd593', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:09.293770 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b6e23-93ef-4256-944f-88e9bb11ae57 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:09.294125 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b6e23-93ef-4256-944f-88e9bb11ae57 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:09.294926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b6e23-93ef-4256-944f-88e9bb11ae57 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] PATH_INFO: `/v3/projects/4207571a0d6348c9b99a380f8df70b6a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:09.296080 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b6e23-93ef-4256-944f-88e9bb11ae57 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4207571a0d6348c9b99a380f8df70b6a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:09.298085 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40b6e23-93ef-4256-944f-88e9bb11ae57 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:09.412544 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d40b6e23-93ef-4256-944f-88e9bb11ae57 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4207571a0d6348c9b99a380f8df70b6a', 'request_id': 'req-d40b6e23-93ef-4256-944f-88e9bb11ae57'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:09.474428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d40b6e23-93ef-4256-944f-88e9bb11ae57 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4207571a0d6348c9b99a380f8df70b6a', 'request_id': 'req-d40b6e23-93ef-4256-944f-88e9bb11ae57'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:09.486209 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d40b6e23-93ef-4256-944f-88e9bb11ae57 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4207571a0d6348c9b99a380f8df70b6a', 'request_id': 'req-d40b6e23-93ef-4256-944f-88e9bb11ae57'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:09.494683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d40b6e23-93ef-4256-944f-88e9bb11ae57 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] The token cache is being invalidate because project 4207571a0d6348c9b99a380f8df70b6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:09.495081 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d40b6e23-93ef-4256-944f-88e9bb11ae57 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:09.495806 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d40b6e23-93ef-4256-944f-88e9bb11ae57 tempest-VolumeQuotaClassesTest-2005497816 tempest-VolumeQuotaClassesTest-2005497816-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:09.497788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1288/2539] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:09 2022] DELETE /identity/v3/projects/4207571a0d6348c9b99a380f8df70b6a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:09.668005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1379186-92b2-4900-b6ed-997de18613c5 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:09.669792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c1379186-92b2-4900-b6ed-997de18613c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:09.742500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1379186-92b2-4900-b6ed-997de18613c5 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:09.804146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b7f48-8767-44d5-a3a6-92f684becaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:09.804771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b7f48-8767-44d5-a3a6-92f684becaf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:09.805052 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b7f48-8767-44d5-a3a6-92f684becaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:09.805322 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b7f48-8767-44d5-a3a6-92f684becaf7 admin admin] PATH_INFO: `/v3/projects/b70cf49189954c638b9e19e3f06aa063` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:09.806456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b7f48-8767-44d5-a3a6-92f684becaf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b70cf49189954c638b9e19e3f06aa063)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:09.808386 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b7f48-8767-44d5-a3a6-92f684becaf7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:09.909471 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:09.913436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fa5b7f48-8767-44d5-a3a6-92f684becaf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b70cf49189954c638b9e19e3f06aa063', 'request_id': 'req-fa5b7f48-8767-44d5-a3a6-92f684becaf7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:09.918839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fa5b7f48-8767-44d5-a3a6-92f684becaf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b70cf49189954c638b9e19e3f06aa063', 'request_id': 'req-fa5b7f48-8767-44d5-a3a6-92f684becaf7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:09.926279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fa5b7f48-8767-44d5-a3a6-92f684becaf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b70cf49189954c638b9e19e3f06aa063', 'request_id': 'req-fa5b7f48-8767-44d5-a3a6-92f684becaf7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:10.000306 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fa5b7f48-8767-44d5-a3a6-92f684becaf7 admin admin] The token cache is being invalidate because project b70cf49189954c638b9e19e3f06aa063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:10.000604 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fa5b7f48-8767-44d5-a3a6-92f684becaf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:10.002072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fa5b7f48-8767-44d5-a3a6-92f684becaf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:10.003410 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1252/2540] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:09 2022] DELETE /identity/v3/projects/b70cf49189954c638b9e19e3f06aa063 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:10.008370 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fcb2e-ba91-4056-9732-247e73e4baf0 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:10.008844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fcb2e-ba91-4056-9732-247e73e4baf0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:10.008966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fcb2e-ba91-4056-9732-247e73e4baf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:10.029286 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-969fcb2e-ba91-4056-9732-247e73e4baf0 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:10.030504 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-969fcb2e-ba91-4056-9732-247e73e4baf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:10.089772 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1289/2541] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:10.098918 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1bdb3-ad9a-42a2-900e-0f0e388452cc None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:10.099218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1bdb3-ad9a-42a2-900e-0f0e388452cc None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:10.099667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1bdb3-ad9a-42a2-900e-0f0e388452cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:10.115586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fcb2e-ba91-4056-9732-247e73e4baf0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:10.117024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-969fcb2e-ba91-4056-9732-247e73e4baf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:10.120345 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-cdc1bdb3-ad9a-42a2-900e-0f0e388452cc None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:10.121520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-cdc1bdb3-ad9a-42a2-900e-0f0e388452cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:10.130144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fcb2e-ba91-4056-9732-247e73e4baf0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:10.131803 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1253/2542] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:10.134487 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1bdb3-ad9a-42a2-900e-0f0e388452cc None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:10.136577 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9bb9f-cf23-4d1b-8de1-2b3b97f24e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:10.137206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9bb9f-cf23-4d1b-8de1-2b3b97f24e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:10.137481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9bb9f-cf23-4d1b-8de1-2b3b97f24e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:10.137757 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9bb9f-cf23-4d1b-8de1-2b3b97f24e38 admin admin] PATH_INFO: `/v3/users/90bccc082ccc4358800399bea08dd3cd` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:10.138969 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9bb9f-cf23-4d1b-8de1-2b3b97f24e38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90bccc082ccc4358800399bea08dd3cd)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:10.140951 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9bb9f-cf23-4d1b-8de1-2b3b97f24e38 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:10.143974 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1bdb3-ad9a-42a2-900e-0f0e388452cc None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:10.149636 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98df64c-7331-4668-8af7-b95624aa1a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:10.150205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98df64c-7331-4668-8af7-b95624aa1a7d admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:10.150475 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98df64c-7331-4668-8af7-b95624aa1a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:10.150741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98df64c-7331-4668-8af7-b95624aa1a7d admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:10.151401 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98df64c-7331-4668-8af7-b95624aa1a7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:08:10.151728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98df64c-7331-4668-8af7-b95624aa1a7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:10.153304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98df64c-7331-4668-8af7-b95624aa1a7d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:10.165490 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1254/2543] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:08:10.170403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98df64c-7331-4668-8af7-b95624aa1a7d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:10.180328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98df64c-7331-4668-8af7-b95624aa1a7d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:10.186229 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa2b8f-671d-4223-8abf-a1078419d5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:10.186824 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa2b8f-671d-4223-8abf-a1078419d5a5 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:10.187153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa2b8f-671d-4223-8abf-a1078419d5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:10.187425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa2b8f-671d-4223-8abf-a1078419d5a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:10.188037 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa2b8f-671d-4223-8abf-a1078419d5a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:10.189904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa2b8f-671d-4223-8abf-a1078419d5a5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:10.320122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1255/2544] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:10 2022] POST /identity/v3/projects => generated 381 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:10.324443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa2b8f-671d-4223-8abf-a1078419d5a5 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:10.385676 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-45c9bb9f-cf23-4d1b-8de1-2b3b97f24e38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90bccc082ccc4358800399bea08dd3cd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:10.386548 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa2b8f-671d-4223-8abf-a1078419d5a5 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:10.391254 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa549ce-2b7e-43cb-bca3-e5989d6bbd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:10.391847 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa549ce-2b7e-43cb-bca3-e5989d6bbd3f admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:10.392122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa549ce-2b7e-43cb-bca3-e5989d6bbd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:10.392440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa549ce-2b7e-43cb-bca3-e5989d6bbd3f admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:10.393051 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa549ce-2b7e-43cb-bca3-e5989d6bbd3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:10.395008 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa549ce-2b7e-43cb-bca3-e5989d6bbd3f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:10.445467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-45c9bb9f-cf23-4d1b-8de1-2b3b97f24e38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90bccc082ccc4358800399bea08dd3cd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:10.446028 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-45c9bb9f-cf23-4d1b-8de1-2b3b97f24e38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90bccc082ccc4358800399bea08dd3cd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:10.448563 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:10.454270 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-45c9bb9f-cf23-4d1b-8de1-2b3b97f24e38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90bccc082ccc4358800399bea08dd3cd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:10.467201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-45c9bb9f-cf23-4d1b-8de1-2b3b97f24e38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90bccc082ccc4358800399bea08dd3cd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:10.473938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1290/2545] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:10 2022] DELETE /identity/v3/users/90bccc082ccc4358800399bea08dd3cd => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:10.478008 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1256/2546] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:10 2022] POST /identity/v3/users => generated 291 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:10.479019 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c41eb-7425-43e9-aec3-3326fcc450ef None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:10.479416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c41eb-7425-43e9-aec3-3326fcc450ef None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:10.479739 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c41eb-7425-43e9-aec3-3326fcc450ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:10.483179 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa549ce-2b7e-43cb-bca3-e5989d6bbd3f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:10.484557 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7aa549ce-2b7e-43cb-bca3-e5989d6bbd3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:10.499972 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa549ce-2b7e-43cb-bca3-e5989d6bbd3f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:10.500665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a44c41eb-7425-43e9-aec3-3326fcc450ef None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:10.502195 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a44c41eb-7425-43e9-aec3-3326fcc450ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:10.566755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beabd23-5e99-475c-90ad-34a2d97146c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:10.567423 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beabd23-5e99-475c-90ad-34a2d97146c7 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:10.567794 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beabd23-5e99-475c-90ad-34a2d97146c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:10.567957 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beabd23-5e99-475c-90ad-34a2d97146c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:10.568585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beabd23-5e99-475c-90ad-34a2d97146c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:10.570176 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beabd23-5e99-475c-90ad-34a2d97146c7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:10.576813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1291/2547] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:10.584243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1257/2548] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:10.588037 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beabd23-5e99-475c-90ad-34a2d97146c7 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:10.588366 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c41eb-7425-43e9-aec3-3326fcc450ef None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:10.589738 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a44c41eb-7425-43e9-aec3-3326fcc450ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:10.599746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beabd23-5e99-475c-90ad-34a2d97146c7 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:10.604610 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76bf25-113e-4a41-92a5-dc1cd721fa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:10.605283 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76bf25-113e-4a41-92a5-dc1cd721fa45 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:10.605623 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76bf25-113e-4a41-92a5-dc1cd721fa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:10.605964 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76bf25-113e-4a41-92a5-dc1cd721fa45 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:10.607205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76bf25-113e-4a41-92a5-dc1cd721fa45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:10.608878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be76bf25-113e-4a41-92a5-dc1cd721fa45 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:10.626100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1258/2549] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:10.627091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76bf25-113e-4a41-92a5-dc1cd721fa45 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:10.633668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:10.639754 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76bf25-113e-4a41-92a5-dc1cd721fa45 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:10.646234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06ba21-be44-4588-bbb8-ffe0c8610d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:10.647316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06ba21-be44-4588-bbb8-ffe0c8610d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:10.647602 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06ba21-be44-4588-bbb8-ffe0c8610d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:10.648126 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06ba21-be44-4588-bbb8-ffe0c8610d7c admin admin] PATH_INFO: `/v3/projects/06457217660e46d38717f33df8843bc8/users/a0cb3cde52ac48e89c7211ced51ce527/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:10.660487 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c41eb-7425-43e9-aec3-3326fcc450ef None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:10.664097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f83382-fefd-4d6e-997f-42786eabbc3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:10.664620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06ba21-be44-4588-bbb8-ffe0c8610d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=06457217660e46d38717f33df8843bc8, user_id=a0cb3cde52ac48e89c7211ced51ce527, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:10.664743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f83382-fefd-4d6e-997f-42786eabbc3f service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:10.665024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f83382-fefd-4d6e-997f-42786eabbc3f service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:10.665303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f83382-fefd-4d6e-997f-42786eabbc3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:10.667766 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06ba21-be44-4588-bbb8-ffe0c8610d7c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:10.675711 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f83382-fefd-4d6e-997f-42786eabbc3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:10.677202 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f83382-fefd-4d6e-997f-42786eabbc3f service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:10.688030 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1292/2550] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:10.764450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1259/2551] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:10 2022] PUT /identity/v3/projects/06457217660e46d38717f33df8843bc8/users/a0cb3cde52ac48e89c7211ced51ce527/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:10.775902 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41847c56-afa3-4191-b959-4e738afdd859 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:10.776482 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41847c56-afa3-4191-b959-4e738afdd859 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:10.776759 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41847c56-afa3-4191-b959-4e738afdd859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:10.806400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-41847c56-afa3-4191-b959-4e738afdd859 None None] MFA Rules not processed for user `a0cb3cde52ac48e89c7211ced51ce527`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:10.807748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-41847c56-afa3-4191-b959-4e738afdd859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:10.874540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1293/2552] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:10.879298 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06ba21-be44-4588-bbb8-ffe0c8610d7c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:10.948981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06ba21-be44-4588-bbb8-ffe0c8610d7c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:10.953780 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb40592-156b-42ad-a351-24a4d3e1e2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:10.954312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb40592-156b-42ad-a351-24a4d3e1e2cf admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:10.954587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb40592-156b-42ad-a351-24a4d3e1e2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:10.954856 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb40592-156b-42ad-a351-24a4d3e1e2cf admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:10.955537 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb40592-156b-42ad-a351-24a4d3e1e2cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:10.957424 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb40592-156b-42ad-a351-24a4d3e1e2cf admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:11.073425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1260/2553] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:10 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:11.077557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41847c56-afa3-4191-b959-4e738afdd859 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:11.088296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41847c56-afa3-4191-b959-4e738afdd859 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:11.092769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:11.104373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb40592-156b-42ad-a351-24a4d3e1e2cf admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:11.106202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ebb40592-156b-42ad-a351-24a4d3e1e2cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:11.164996 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9969a95-90be-4652-8f42-cfe939146de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:11.165752 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9969a95-90be-4652-8f42-cfe939146de7 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:11.166034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9969a95-90be-4652-8f42-cfe939146de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:11.166304 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9969a95-90be-4652-8f42-cfe939146de7 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:11.167578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9969a95-90be-4652-8f42-cfe939146de7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:11.170282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9969a95-90be-4652-8f42-cfe939146de7 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:11.185792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb40592-156b-42ad-a351-24a4d3e1e2cf admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:11.190752 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:11.191419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:11.191701 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:11.191974 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4 admin admin] PATH_INFO: `/v3/projects/58e42f40ea83476cafb97c10bb4f4996` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:11.193107 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58e42f40ea83476cafb97c10bb4f4996)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:11.195124 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:11.295672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1294/2554] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:11 2022] POST /identity/v3/users => generated 296 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:11.299938 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9969a95-90be-4652-8f42-cfe939146de7 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:11.312143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9969a95-90be-4652-8f42-cfe939146de7 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:11.312455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58e42f40ea83476cafb97c10bb4f4996', 'request_id': 'req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:11.320488 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58e42f40ea83476cafb97c10bb4f4996', 'request_id': 'req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:11.327878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58e42f40ea83476cafb97c10bb4f4996', 'request_id': 'req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:11.333531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:11.372848 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa153d17-aa89-4eac-91e6-08ab4a6b7871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:11.373185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa153d17-aa89-4eac-91e6-08ab4a6b7871 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:11.373466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa153d17-aa89-4eac-91e6-08ab4a6b7871 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:11.373735 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa153d17-aa89-4eac-91e6-08ab4a6b7871 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:11.374387 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa153d17-aa89-4eac-91e6-08ab4a6b7871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:11.376331 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa153d17-aa89-4eac-91e6-08ab4a6b7871 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:11.386547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1295/2555] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:11.391566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa153d17-aa89-4eac-91e6-08ab4a6b7871 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:11.399132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4 admin admin] The token cache is being invalidate because project 58e42f40ea83476cafb97c10bb4f4996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:11.399456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:11.400198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:11.401986 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1261/2556] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:11 2022] DELETE /identity/v3/projects/58e42f40ea83476cafb97c10bb4f4996 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:11.402113 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa153d17-aa89-4eac-91e6-08ab4a6b7871 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:11.406290 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:11.407732 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:11.408891 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d23ed7-5657-4b7f-8bf3-6f87625ff2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:11.409553 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d23ed7-5657-4b7f-8bf3-6f87625ff2e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:11.409839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d23ed7-5657-4b7f-8bf3-6f87625ff2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:11.410114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d23ed7-5657-4b7f-8bf3-6f87625ff2e5 admin admin] PATH_INFO: `/v3/projects/9749a26f541c4e8faed611aff675972d/users/32bf0639aeca4c649b5255a43970846b/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:11.424739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df0ba6-66dd-4e87-a6b7-420aeb2893e4 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:11.427801 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d23ed7-5657-4b7f-8bf3-6f87625ff2e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9749a26f541c4e8faed611aff675972d, user_id=32bf0639aeca4c649b5255a43970846b, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:11.430043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc247bb2-d548-4b0d-8e27-7cca34a2a6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:11.430868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc247bb2-d548-4b0d-8e27-7cca34a2a6a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:11.431049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d23ed7-5657-4b7f-8bf3-6f87625ff2e5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:11.431279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc247bb2-d548-4b0d-8e27-7cca34a2a6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:11.431598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc247bb2-d548-4b0d-8e27-7cca34a2a6a0 admin admin] PATH_INFO: `/v3/users/4e82ff8c72f24d11b542cb43b1ade2ab` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:11.432852 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc247bb2-d548-4b0d-8e27-7cca34a2a6a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e82ff8c72f24d11b542cb43b1ade2ab)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:11.435111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc247bb2-d548-4b0d-8e27-7cca34a2a6a0 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:11.515330 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1296/2557] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:11 2022] PUT /identity/v3/projects/9749a26f541c4e8faed611aff675972d/users/32bf0639aeca4c649b5255a43970846b/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:11.519233 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d23ed7-5657-4b7f-8bf3-6f87625ff2e5 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:11.520824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-22d23ed7-5657-4b7f-8bf3-6f87625ff2e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:11.588085 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d23ed7-5657-4b7f-8bf3-6f87625ff2e5 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:11.592842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4d447-e9fc-4772-b42d-8b61bc9495c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:11.593443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4d447-e9fc-4772-b42d-8b61bc9495c9 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:11.593723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4d447-e9fc-4772-b42d-8b61bc9495c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:11.593993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4d447-e9fc-4772-b42d-8b61bc9495c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:11.594631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4d447-e9fc-4772-b42d-8b61bc9495c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:11.596458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4d447-e9fc-4772-b42d-8b61bc9495c9 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:11.607316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1297/2558] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:11.613179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4d447-e9fc-4772-b42d-8b61bc9495c9 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:11.620426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:11.628955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cc247bb2-d548-4b0d-8e27-7cca34a2a6a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e82ff8c72f24d11b542cb43b1ade2ab'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:11.633566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4d447-e9fc-4772-b42d-8b61bc9495c9 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:11.638803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909055f-baa4-4baf-bf12-87c080065d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:11.639514 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909055f-baa4-4baf-bf12-87c080065d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:11.639796 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909055f-baa4-4baf-bf12-87c080065d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:11.640072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909055f-baa4-4baf-bf12-87c080065d0d admin admin] PATH_INFO: `/v3/projects/9749a26f541c4e8faed611aff675972d/users/32bf0639aeca4c649b5255a43970846b/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:11.642068 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909055f-baa4-4baf-bf12-87c080065d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9749a26f541c4e8faed611aff675972d, user_id=32bf0639aeca4c649b5255a43970846b, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:11.644885 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909055f-baa4-4baf-bf12-87c080065d0d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:11.657121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1298/2559] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:11 2022] PUT /identity/v3/projects/9749a26f541c4e8faed611aff675972d/users/32bf0639aeca4c649b5255a43970846b/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:11.668021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd9aab-5526-474f-8844-a5893fe5ca87 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:11.668402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd9aab-5526-474f-8844-a5893fe5ca87 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:11.668772 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd9aab-5526-474f-8844-a5893fe5ca87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:11.702417 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-26fd9aab-5526-474f-8844-a5893fe5ca87 None None] MFA Rules not processed for user `32bf0639aeca4c649b5255a43970846b`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:11.703994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-26fd9aab-5526-474f-8844-a5893fe5ca87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:11.726174 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cc247bb2-d548-4b0d-8e27-7cca34a2a6a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e82ff8c72f24d11b542cb43b1ade2ab'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:11.745117 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cc247bb2-d548-4b0d-8e27-7cca34a2a6a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e82ff8c72f24d11b542cb43b1ade2ab'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:11.745885 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cc247bb2-d548-4b0d-8e27-7cca34a2a6a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e82ff8c72f24d11b542cb43b1ade2ab'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:11.761484 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cc247bb2-d548-4b0d-8e27-7cca34a2a6a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e82ff8c72f24d11b542cb43b1ade2ab'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:11.774593 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1262/2560] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:11 2022] DELETE /identity/v3/users/4e82ff8c72f24d11b542cb43b1ade2ab => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:11.815902 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1299/2561] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:11.824085 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc247bb2-d548-4b0d-8e27-7cca34a2a6a0 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:11.826526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-cc247bb2-d548-4b0d-8e27-7cca34a2a6a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:11.894967 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc247bb2-d548-4b0d-8e27-7cca34a2a6a0 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:11.899117 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e860a3-56c4-4e6e-9bad-7917314f7183 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:11.899729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e860a3-56c4-4e6e-9bad-7917314f7183 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:11.900006 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e860a3-56c4-4e6e-9bad-7917314f7183 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:11.900268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e860a3-56c4-4e6e-9bad-7917314f7183 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:11.961803 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e860a3-56c4-4e6e-9bad-7917314f7183 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:11.963291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e860a3-56c4-4e6e-9bad-7917314f7183 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:11.974002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1263/2562] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:12.087906 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd9aab-5526-474f-8844-a5893fe5ca87 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:12.123557 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e860a3-56c4-4e6e-9bad-7917314f7183 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:12.125876 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-16e860a3-56c4-4e6e-9bad-7917314f7183 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:12.167157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:12.179470 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd9aab-5526-474f-8844-a5893fe5ca87 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:12.184128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2818c-fd35-4544-874a-b554933fed35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:12.184891 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2818c-fd35-4544-874a-b554933fed35 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:12.185172 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2818c-fd35-4544-874a-b554933fed35 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:12.185474 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2818c-fd35-4544-874a-b554933fed35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:12.187473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-44d2818c-fd35-4544-874a-b554933fed35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:12.262096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e860a3-56c4-4e6e-9bad-7917314f7183 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:12.269819 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f0a7a-b532-4b9a-9ed0-21d2e3b77bc0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9506139d5744fe69153e7e30a05b436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:12.270425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f0a7a-b532-4b9a-9ed0-21d2e3b77bc0 service placement] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:12.270562 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d2818c-fd35-4544-874a-b554933fed35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:12.270706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f0a7a-b532-4b9a-9ed0-21d2e3b77bc0 service placement] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:12.271013 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f0a7a-b532-4b9a-9ed0-21d2e3b77bc0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:12.271913 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d2818c-fd35-4544-874a-b554933fed35 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:12.273125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b99f0a7a-b532-4b9a-9ed0-21d2e3b77bc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:12.286528 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f0a7a-b532-4b9a-9ed0-21d2e3b77bc0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:12.286864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1300/2563] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:12.287695 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2818c-fd35-4544-874a-b554933fed35 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:12.288935 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-44d2818c-fd35-4544-874a-b554933fed35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:12.289782 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f0a7a-b532-4b9a-9ed0-21d2e3b77bc0 service placement] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:12.302569 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1264/2564] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:08:12.327363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:12.361710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2818c-fd35-4544-874a-b554933fed35 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:12.366682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf67d8-1054-4955-8747-9b0d52afb51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:12.367354 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf67d8-1054-4955-8747-9b0d52afb51a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:12.369389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf67d8-1054-4955-8747-9b0d52afb51a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:12.369389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf67d8-1054-4955-8747-9b0d52afb51a admin admin] PATH_INFO: `/v3/projects/78e9dacfab33473fac2c930c13dfd593` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:12.369389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf67d8-1054-4955-8747-9b0d52afb51a admin admin] RBAC: Authorizing `identity:delete_project(project_id=78e9dacfab33473fac2c930c13dfd593)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:12.371006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf67d8-1054-4955-8747-9b0d52afb51a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:12.519258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ccbf67d8-1054-4955-8747-9b0d52afb51a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78e9dacfab33473fac2c930c13dfd593', 'request_id': 'req-ccbf67d8-1054-4955-8747-9b0d52afb51a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:12.594966 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ccbf67d8-1054-4955-8747-9b0d52afb51a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78e9dacfab33473fac2c930c13dfd593', 'request_id': 'req-ccbf67d8-1054-4955-8747-9b0d52afb51a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:12.605924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ccbf67d8-1054-4955-8747-9b0d52afb51a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78e9dacfab33473fac2c930c13dfd593', 'request_id': 'req-ccbf67d8-1054-4955-8747-9b0d52afb51a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:12.619525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ccbf67d8-1054-4955-8747-9b0d52afb51a admin admin] The token cache is being invalidate because project 78e9dacfab33473fac2c930c13dfd593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:12.619809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ccbf67d8-1054-4955-8747-9b0d52afb51a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:12.620477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ccbf67d8-1054-4955-8747-9b0d52afb51a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:12.622021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1301/2565] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:12 2022] DELETE /identity/v3/projects/78e9dacfab33473fac2c930c13dfd593 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:12.627989 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d3db3-b21d-4974-8d1f-8cfb7209cbe4 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:12.628280 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d3db3-b21d-4974-8d1f-8cfb7209cbe4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:12.628561 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d3db3-b21d-4974-8d1f-8cfb7209cbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:12.648905 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a05d3db3-b21d-4974-8d1f-8cfb7209cbe4 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:12.650170 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a05d3db3-b21d-4974-8d1f-8cfb7209cbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:12.710123 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1265/2566] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:12.719110 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ed468-c0ce-4864-9140-ec03004bac4f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:12.719397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ed468-c0ce-4864-9140-ec03004bac4f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:12.719674 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ed468-c0ce-4864-9140-ec03004bac4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:12.739344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-635ed468-c0ce-4864-9140-ec03004bac4f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:12.740576 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-635ed468-c0ce-4864-9140-ec03004bac4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:12.749365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1302/2567] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:12.753903 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d3db3-b21d-4974-8d1f-8cfb7209cbe4 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:12.764328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d3db3-b21d-4974-8d1f-8cfb7209cbe4 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:12.768983 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c31419-00ff-4abb-8a85-c42aed1291cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:12.769629 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c31419-00ff-4abb-8a85-c42aed1291cd admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:12.769907 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c31419-00ff-4abb-8a85-c42aed1291cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:12.770179 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c31419-00ff-4abb-8a85-c42aed1291cd admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:12.770810 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c31419-00ff-4abb-8a85-c42aed1291cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:08:12.771141 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c31419-00ff-4abb-8a85-c42aed1291cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:12.772981 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c31419-00ff-4abb-8a85-c42aed1291cd admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:12.789774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1266/2568] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:08:12.791226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ed468-c0ce-4864-9140-ec03004bac4f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:12.801524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ed468-c0ce-4864-9140-ec03004bac4f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:12.812164 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fdecb8-fba9-4429-9ab5-4b408ab511bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:12.812164 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdecb8-fba9-4429-9ab5-4b408ab511bc admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:12.812587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdecb8-fba9-4429-9ab5-4b408ab511bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:12.812931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdecb8-fba9-4429-9ab5-4b408ab511bc admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:12.813572 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fdecb8-fba9-4429-9ab5-4b408ab511bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:12.815517 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fdecb8-fba9-4429-9ab5-4b408ab511bc admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:12.921973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1303/2569] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:12 2022] POST /identity/v3/projects => generated 389 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:12.922732 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c31419-00ff-4abb-8a85-c42aed1291cd admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:12.943956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c31419-00ff-4abb-8a85-c42aed1291cd admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:13.015793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9af60-4c29-4fb0-a16c-353958ce3466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:13.017427 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9af60-4c29-4fb0-a16c-353958ce3466 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:13.017708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9af60-4c29-4fb0-a16c-353958ce3466 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:13.017982 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9af60-4c29-4fb0-a16c-353958ce3466 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:13.018620 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9af60-4c29-4fb0-a16c-353958ce3466 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:13.021631 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9af60-4c29-4fb0-a16c-353958ce3466 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:13.118923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1267/2570] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:12 2022] POST /identity/v3/users => generated 295 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:13.121819 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fdecb8-fba9-4429-9ab5-4b408ab511bc admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:13.133354 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fdecb8-fba9-4429-9ab5-4b408ab511bc admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:13.138126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5372f0-8f03-4711-b49b-6194b9ffb5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:13.138726 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5372f0-8f03-4711-b49b-6194b9ffb5aa admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:13.139042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5372f0-8f03-4711-b49b-6194b9ffb5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:13.139300 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5372f0-8f03-4711-b49b-6194b9ffb5aa admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:13.139939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5372f0-8f03-4711-b49b-6194b9ffb5aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:13.141511 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5372f0-8f03-4711-b49b-6194b9ffb5aa admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:13.151959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1304/2571] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:13.156070 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9af60-4c29-4fb0-a16c-353958ce3466 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:13.162608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:13.166700 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9af60-4c29-4fb0-a16c-353958ce3466 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:13.172071 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e754324-e41e-476f-8fcb-0af124016ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:13.173317 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e754324-e41e-476f-8fcb-0af124016ffe admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:13.173317 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e754324-e41e-476f-8fcb-0af124016ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:13.173317 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e754324-e41e-476f-8fcb-0af124016ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:13.173886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e754324-e41e-476f-8fcb-0af124016ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:13.175537 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e754324-e41e-476f-8fcb-0af124016ffe admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:13.186753 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1268/2572] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:13.191178 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5372f0-8f03-4711-b49b-6194b9ffb5aa admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:13.201265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5372f0-8f03-4711-b49b-6194b9ffb5aa admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:13.205863 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af63a7a-81de-4fe3-8bad-2bd72b8ff850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:13.206511 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af63a7a-81de-4fe3-8bad-2bd72b8ff850 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:13.206805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af63a7a-81de-4fe3-8bad-2bd72b8ff850 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:13.207125 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af63a7a-81de-4fe3-8bad-2bd72b8ff850 admin admin] PATH_INFO: `/v3/projects/1a2d5c93f0474c668a8a927f225cd8a3/users/24049f5b17ad4c0ca639ebc46a61cb62/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:13.223241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af63a7a-81de-4fe3-8bad-2bd72b8ff850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a2d5c93f0474c668a8a927f225cd8a3, user_id=24049f5b17ad4c0ca639ebc46a61cb62, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:13.226210 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af63a7a-81de-4fe3-8bad-2bd72b8ff850 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:13.342079 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1305/2573] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:13 2022] PUT /identity/v3/projects/1a2d5c93f0474c668a8a927f225cd8a3/users/24049f5b17ad4c0ca639ebc46a61cb62/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:13.352350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fdef1-14bb-4b82-8411-3e6e838287f9 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:13.352714 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fdef1-14bb-4b82-8411-3e6e838287f9 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:13.353043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fdef1-14bb-4b82-8411-3e6e838287f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:13.382095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9e8fdef1-14bb-4b82-8411-3e6e838287f9 None None] MFA Rules not processed for user `24049f5b17ad4c0ca639ebc46a61cb62`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:13.383341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9e8fdef1-14bb-4b82-8411-3e6e838287f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:13.449218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1269/2574] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:13.458252 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af63a7a-81de-4fe3-8bad-2bd72b8ff850 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:13.523736 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:13.527462 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af63a7a-81de-4fe3-8bad-2bd72b8ff850 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:13.532828 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c705d-7163-4a9d-a892-ba472fec284d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:13.533433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c705d-7163-4a9d-a892-ba472fec284d admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:13.533714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c705d-7163-4a9d-a892-ba472fec284d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:13.533993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c705d-7163-4a9d-a892-ba472fec284d admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:13.534630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c705d-7163-4a9d-a892-ba472fec284d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:13.536579 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c705d-7163-4a9d-a892-ba472fec284d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:13.647528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1306/2575] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:13 2022] POST /identity/v3/projects => generated 389 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:13.651951 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fdef1-14bb-4b82-8411-3e6e838287f9 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:13.663874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fdef1-14bb-4b82-8411-3e6e838287f9 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:13.723077 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f67fe-cb10-4e9d-80f0-52a36bfc78db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:13.723681 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f67fe-cb10-4e9d-80f0-52a36bfc78db admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:13.723959 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f67fe-cb10-4e9d-80f0-52a36bfc78db admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:13.724231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f67fe-cb10-4e9d-80f0-52a36bfc78db admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:13.724865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f67fe-cb10-4e9d-80f0-52a36bfc78db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:13.727277 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f67fe-cb10-4e9d-80f0-52a36bfc78db admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:13.837462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1270/2576] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:13 2022] POST /identity/v3/users => generated 301 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:13.842621 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c705d-7163-4a9d-a892-ba472fec284d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:13.853384 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c705d-7163-4a9d-a892-ba472fec284d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:13.857979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ade773-b682-4b02-9bfa-825e6b2a12ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:13.858654 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ade773-b682-4b02-9bfa-825e6b2a12ae admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:13.859334 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ade773-b682-4b02-9bfa-825e6b2a12ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:13.859818 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ade773-b682-4b02-9bfa-825e6b2a12ae admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:13.860548 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ade773-b682-4b02-9bfa-825e6b2a12ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:13.862381 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ade773-b682-4b02-9bfa-825e6b2a12ae admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:13.872404 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1307/2577] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:13.877199 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f67fe-cb10-4e9d-80f0-52a36bfc78db admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:13.887282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f67fe-cb10-4e9d-80f0-52a36bfc78db admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:13.892869 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bb304-2c8f-4d77-9ad6-c1a6bf0cbd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:13.893559 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bb304-2c8f-4d77-9ad6-c1a6bf0cbd03 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:13.893935 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bb304-2c8f-4d77-9ad6-c1a6bf0cbd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:13.894265 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6bb304-2c8f-4d77-9ad6-c1a6bf0cbd03 admin admin] PATH_INFO: `/v3/projects/858b9c3296554422855b97ca1f8629c0/users/1a260a20c5ce4a74aa1ffc3801201674/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:13.911800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6bb304-2c8f-4d77-9ad6-c1a6bf0cbd03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=858b9c3296554422855b97ca1f8629c0, user_id=1a260a20c5ce4a74aa1ffc3801201674, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:13.914463 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6bb304-2c8f-4d77-9ad6-c1a6bf0cbd03 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:14.018690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1271/2578] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:13 2022] PUT /identity/v3/projects/858b9c3296554422855b97ca1f8629c0/users/1a260a20c5ce4a74aa1ffc3801201674/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:14.023088 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ade773-b682-4b02-9bfa-825e6b2a12ae admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:14.033812 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ade773-b682-4b02-9bfa-825e6b2a12ae admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:14.099091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71aea6a-3ccc-4169-a8b1-1321d794d693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:14.100021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71aea6a-3ccc-4169-a8b1-1321d794d693 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:14.100021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71aea6a-3ccc-4169-a8b1-1321d794d693 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:14.100270 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71aea6a-3ccc-4169-a8b1-1321d794d693 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:14.100887 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71aea6a-3ccc-4169-a8b1-1321d794d693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:14.102466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71aea6a-3ccc-4169-a8b1-1321d794d693 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:14.121501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1308/2579] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:14.124495 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bb304-2c8f-4d77-9ad6-c1a6bf0cbd03 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:14.139112 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6bb304-2c8f-4d77-9ad6-c1a6bf0cbd03 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:14.151728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf8e45-54c1-41ac-981b-88182a09a273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:14.151728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf8e45-54c1-41ac-981b-88182a09a273 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:14.151728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf8e45-54c1-41ac-981b-88182a09a273 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:14.151728 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf8e45-54c1-41ac-981b-88182a09a273 admin admin] PATH_INFO: `/v3/projects/858b9c3296554422855b97ca1f8629c0/users/1a260a20c5ce4a74aa1ffc3801201674/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:14.152377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf8e45-54c1-41ac-981b-88182a09a273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=858b9c3296554422855b97ca1f8629c0, user_id=1a260a20c5ce4a74aa1ffc3801201674, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:14.152377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf8e45-54c1-41ac-981b-88182a09a273 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:14.161938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1272/2580] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:14 2022] PUT /identity/v3/projects/858b9c3296554422855b97ca1f8629c0/users/1a260a20c5ce4a74aa1ffc3801201674/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:14.172434 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7d8aa-6714-41e0-8866-69450dc7b247 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:14.172955 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7d8aa-6714-41e0-8866-69450dc7b247 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:14.173041 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7d8aa-6714-41e0-8866-69450dc7b247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:14.202289 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-cff7d8aa-6714-41e0-8866-69450dc7b247 None None] MFA Rules not processed for user `1a260a20c5ce4a74aa1ffc3801201674`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:14.203526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-cff7d8aa-6714-41e0-8866-69450dc7b247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:14.281487 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1309/2581] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:14 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:14.292201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf8e45-54c1-41ac-981b-88182a09a273 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:14.294395 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-39bf8e45-54c1-41ac-981b-88182a09a273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:14.336132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:14.353003 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf8e45-54c1-41ac-981b-88182a09a273 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:14.357254 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3597f-8e92-417c-940c-032973ec14f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:14.358063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3597f-8e92-417c-940c-032973ec14f8 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:14.358386 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3597f-8e92-417c-940c-032973ec14f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:14.358646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3597f-8e92-417c-940c-032973ec14f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:14.367621 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3597f-8e92-417c-940c-032973ec14f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:14.369033 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de3597f-8e92-417c-940c-032973ec14f8 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:14.380652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1273/2582] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:14 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:14.547782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7d8aa-6714-41e0-8866-69450dc7b247 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:14.603568 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:14.615666 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7d8aa-6714-41e0-8866-69450dc7b247 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:14.621192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54653b4f-5889-4a80-b521-1988e7323c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:14.621192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54653b4f-5889-4a80-b521-1988e7323c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:14.621694 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54653b4f-5889-4a80-b521-1988e7323c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:14.621786 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54653b4f-5889-4a80-b521-1988e7323c5e admin admin] PATH_INFO: `/v3/users/24049f5b17ad4c0ca639ebc46a61cb62` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:14.623704 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54653b4f-5889-4a80-b521-1988e7323c5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=24049f5b17ad4c0ca639ebc46a61cb62)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:14.625557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54653b4f-5889-4a80-b521-1988e7323c5e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:14.833920 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-54653b4f-5889-4a80-b521-1988e7323c5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24049f5b17ad4c0ca639ebc46a61cb62'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:14.906282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-54653b4f-5889-4a80-b521-1988e7323c5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24049f5b17ad4c0ca639ebc46a61cb62'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:14.906935 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-54653b4f-5889-4a80-b521-1988e7323c5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24049f5b17ad4c0ca639ebc46a61cb62'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:14.920801 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-54653b4f-5889-4a80-b521-1988e7323c5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24049f5b17ad4c0ca639ebc46a61cb62'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:14.933560 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-54653b4f-5889-4a80-b521-1988e7323c5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24049f5b17ad4c0ca639ebc46a61cb62'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:14.941585 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1310/2583] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:14 2022] DELETE /identity/v3/users/24049f5b17ad4c0ca639ebc46a61cb62 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:14.946247 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab69fb-54fc-45c0-921d-0c5aef15cbae None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:14.946712 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab69fb-54fc-45c0-921d-0c5aef15cbae None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:14.946806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab69fb-54fc-45c0-921d-0c5aef15cbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:14.967846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8bab69fb-54fc-45c0-921d-0c5aef15cbae None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:14.969190 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8bab69fb-54fc-45c0-921d-0c5aef15cbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:15.033727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1274/2584] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:15.043498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54653b4f-5889-4a80-b521-1988e7323c5e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:15.046450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-54653b4f-5889-4a80-b521-1988e7323c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:15.110936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54653b4f-5889-4a80-b521-1988e7323c5e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:15.116246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0850d6-350a-4d4a-bf69-74f34c72e713 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:15.116886 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0850d6-350a-4d4a-bf69-74f34c72e713 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:15.117166 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0850d6-350a-4d4a-bf69-74f34c72e713 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:15.117435 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0850d6-350a-4d4a-bf69-74f34c72e713 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:15.127575 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0850d6-350a-4d4a-bf69-74f34c72e713 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:15.129037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0850d6-350a-4d4a-bf69-74f34c72e713 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:15.140557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1311/2585] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:15.481153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab69fb-54fc-45c0-921d-0c5aef15cbae None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:15.485285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8bab69fb-54fc-45c0-921d-0c5aef15cbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:15.502874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab69fb-54fc-45c0-921d-0c5aef15cbae None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:15.513910 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f92d40-dd65-41da-9ee0-a8e29d550db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:15.514589 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f92d40-dd65-41da-9ee0-a8e29d550db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:15.514934 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f92d40-dd65-41da-9ee0-a8e29d550db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:15.515188 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f92d40-dd65-41da-9ee0-a8e29d550db2 admin admin] PATH_INFO: `/v3/users/a0cb3cde52ac48e89c7211ced51ce527` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:15.516400 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f92d40-dd65-41da-9ee0-a8e29d550db2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0cb3cde52ac48e89c7211ced51ce527)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:15.518451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f92d40-dd65-41da-9ee0-a8e29d550db2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:15.596948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0850d6-350a-4d4a-bf69-74f34c72e713 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:15.601064 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-bc0850d6-350a-4d4a-bf69-74f34c72e713 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:15.617082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0850d6-350a-4d4a-bf69-74f34c72e713 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:15.624778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6b241-29b0-4914-bc31-15e750839bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:15.626717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6b241-29b0-4914-bc31-15e750839bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:15.626717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6b241-29b0-4914-bc31-15e750839bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:15.626717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6b241-29b0-4914-bc31-15e750839bb2 admin admin] PATH_INFO: `/v3/projects/1a2d5c93f0474c668a8a927f225cd8a3` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:15.627905 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6b241-29b0-4914-bc31-15e750839bb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a2d5c93f0474c668a8a927f225cd8a3)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:15.630520 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6b241-29b0-4914-bc31-15e750839bb2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:15.717457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-21f92d40-dd65-41da-9ee0-a8e29d550db2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0cb3cde52ac48e89c7211ced51ce527'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:15.745970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0fb6b241-29b0-4914-bc31-15e750839bb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a2d5c93f0474c668a8a927f225cd8a3', 'request_id': 'req-0fb6b241-29b0-4914-bc31-15e750839bb2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:15.828646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-21f92d40-dd65-41da-9ee0-a8e29d550db2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0cb3cde52ac48e89c7211ced51ce527'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:15.844896 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-21f92d40-dd65-41da-9ee0-a8e29d550db2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0cb3cde52ac48e89c7211ced51ce527'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:15.845627 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-21f92d40-dd65-41da-9ee0-a8e29d550db2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0cb3cde52ac48e89c7211ced51ce527'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:15.853121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-21f92d40-dd65-41da-9ee0-a8e29d550db2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0cb3cde52ac48e89c7211ced51ce527'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:15.861889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1275/2586] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:15 2022] DELETE /identity/v3/users/a0cb3cde52ac48e89c7211ced51ce527 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:15.866287 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e7094-6e09-48fd-ab80-14a570a01f52 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:15.866696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e7094-6e09-48fd-ab80-14a570a01f52 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:15.867007 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e7094-6e09-48fd-ab80-14a570a01f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:15.887525 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f16e7094-6e09-48fd-ab80-14a570a01f52 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:15.889105 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f16e7094-6e09-48fd-ab80-14a570a01f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:15.914834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0fb6b241-29b0-4914-bc31-15e750839bb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a2d5c93f0474c668a8a927f225cd8a3', 'request_id': 'req-0fb6b241-29b0-4914-bc31-15e750839bb2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:15.923972 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0fb6b241-29b0-4914-bc31-15e750839bb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a2d5c93f0474c668a8a927f225cd8a3', 'request_id': 'req-0fb6b241-29b0-4914-bc31-15e750839bb2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:15.933615 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:15.942606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0fb6b241-29b0-4914-bc31-15e750839bb2 admin admin] The token cache is being invalidate because project 1a2d5c93f0474c668a8a927f225cd8a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:15.943217 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0fb6b241-29b0-4914-bc31-15e750839bb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:15.943990 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0fb6b241-29b0-4914-bc31-15e750839bb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:15.946534 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1312/2587] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:15 2022] DELETE /identity/v3/projects/1a2d5c93f0474c668a8a927f225cd8a3 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:15.952312 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6b241-29b0-4914-bc31-15e750839bb2 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:15.954095 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0fb6b241-29b0-4914-bc31-15e750839bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:15.974717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:16.021425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1276/2588] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:16.034814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e7094-6e09-48fd-ab80-14a570a01f52 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:16.036974 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f16e7094-6e09-48fd-ab80-14a570a01f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:16.077591 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6b241-29b0-4914-bc31-15e750839bb2 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:16.084992 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00de599-b04c-4a52-880b-b09b6e8c639d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:16.086432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00de599-b04c-4a52-880b-b09b6e8c639d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:16.086834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00de599-b04c-4a52-880b-b09b6e8c639d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:16.087114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00de599-b04c-4a52-880b-b09b6e8c639d admin admin] PATH_INFO: `/v3/users/1a260a20c5ce4a74aa1ffc3801201674` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:16.088654 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00de599-b04c-4a52-880b-b09b6e8c639d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a260a20c5ce4a74aa1ffc3801201674)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:16.090707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00de599-b04c-4a52-880b-b09b6e8c639d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:16.116077 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e7094-6e09-48fd-ab80-14a570a01f52 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:16.123238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297f503-acf5-4e02-9608-b6e2333c56ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:16.124063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297f503-acf5-4e02-9608-b6e2333c56ac service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:16.124063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297f503-acf5-4e02-9608-b6e2333c56ac service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:16.124256 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297f503-acf5-4e02-9608-b6e2333c56ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:16.135318 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a297f503-acf5-4e02-9608-b6e2333c56ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:16.136766 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a297f503-acf5-4e02-9608-b6e2333c56ac service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:16.147122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1277/2589] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:16.294927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e00de599-b04c-4a52-880b-b09b6e8c639d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a260a20c5ce4a74aa1ffc3801201674'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:16.430980 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e00de599-b04c-4a52-880b-b09b6e8c639d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a260a20c5ce4a74aa1ffc3801201674'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:16.432983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e00de599-b04c-4a52-880b-b09b6e8c639d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a260a20c5ce4a74aa1ffc3801201674'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:16.443098 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e00de599-b04c-4a52-880b-b09b6e8c639d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a260a20c5ce4a74aa1ffc3801201674'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:16.479287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e00de599-b04c-4a52-880b-b09b6e8c639d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a260a20c5ce4a74aa1ffc3801201674'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:16.493292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1313/2590] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:15 2022] DELETE /identity/v3/users/1a260a20c5ce4a74aa1ffc3801201674 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:16.694566 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297f503-acf5-4e02-9608-b6e2333c56ac service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:16.697409 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a297f503-acf5-4e02-9608-b6e2333c56ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:16.718323 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297f503-acf5-4e02-9608-b6e2333c56ac service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:16.822214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbc1ac-6bf5-4543-95f1-e506570bcec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:16.822908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbc1ac-6bf5-4543-95f1-e506570bcec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:16.823212 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbc1ac-6bf5-4543-95f1-e506570bcec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:16.823491 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbc1ac-6bf5-4543-95f1-e506570bcec2 admin admin] PATH_INFO: `/v3/projects/06457217660e46d38717f33df8843bc8` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:16.824793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbc1ac-6bf5-4543-95f1-e506570bcec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06457217660e46d38717f33df8843bc8)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:16.827331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbc1ac-6bf5-4543-95f1-e506570bcec2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:16.943965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-27fbc1ac-6bf5-4543-95f1-e506570bcec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06457217660e46d38717f33df8843bc8', 'request_id': 'req-27fbc1ac-6bf5-4543-95f1-e506570bcec2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:16.951553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-27fbc1ac-6bf5-4543-95f1-e506570bcec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06457217660e46d38717f33df8843bc8', 'request_id': 'req-27fbc1ac-6bf5-4543-95f1-e506570bcec2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:16.959922 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-27fbc1ac-6bf5-4543-95f1-e506570bcec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06457217660e46d38717f33df8843bc8', 'request_id': 'req-27fbc1ac-6bf5-4543-95f1-e506570bcec2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:17.000426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00de599-b04c-4a52-880b-b09b6e8c639d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:17.002973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e00de599-b04c-4a52-880b-b09b6e8c639d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:17.027071 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00de599-b04c-4a52-880b-b09b6e8c639d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:17.064918 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-27fbc1ac-6bf5-4543-95f1-e506570bcec2 admin admin] The token cache is being invalidate because project 06457217660e46d38717f33df8843bc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:17.065399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-27fbc1ac-6bf5-4543-95f1-e506570bcec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:17.066161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-27fbc1ac-6bf5-4543-95f1-e506570bcec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:17.071815 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1278/2591] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:16 2022] DELETE /identity/v3/projects/06457217660e46d38717f33df8843bc8 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:17.073347 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbc1ac-6bf5-4543-95f1-e506570bcec2 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:17.075924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-27fbc1ac-6bf5-4543-95f1-e506570bcec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:17.113382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f26b01-5bbe-433a-942d-db3bbb8f8893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:17.114307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f26b01-5bbe-433a-942d-db3bbb8f8893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:17.114764 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f26b01-5bbe-433a-942d-db3bbb8f8893 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:17.115144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f26b01-5bbe-433a-942d-db3bbb8f8893 admin admin] PATH_INFO: `/v3/projects/858b9c3296554422855b97ca1f8629c0` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:17.116423 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f26b01-5bbe-433a-942d-db3bbb8f8893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=858b9c3296554422855b97ca1f8629c0)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:17.117779 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:17.119625 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f26b01-5bbe-433a-942d-db3bbb8f8893 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:17.150811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:17.181766 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbc1ac-6bf5-4543-95f1-e506570bcec2 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:17.189826 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee9efa-0405-4ddb-84aa-a57f07816824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:17.190661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee9efa-0405-4ddb-84aa-a57f07816824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:17.192201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee9efa-0405-4ddb-84aa-a57f07816824 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:17.192781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee9efa-0405-4ddb-84aa-a57f07816824 admin admin] PATH_INFO: `/v3/users/32bf0639aeca4c649b5255a43970846b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:17.194182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee9efa-0405-4ddb-84aa-a57f07816824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32bf0639aeca4c649b5255a43970846b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:17.196380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee9efa-0405-4ddb-84aa-a57f07816824 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:17.263864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-80f26b01-5bbe-433a-942d-db3bbb8f8893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '858b9c3296554422855b97ca1f8629c0', 'request_id': 'req-80f26b01-5bbe-433a-942d-db3bbb8f8893'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:17.435371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-80f26b01-5bbe-433a-942d-db3bbb8f8893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '858b9c3296554422855b97ca1f8629c0', 'request_id': 'req-80f26b01-5bbe-433a-942d-db3bbb8f8893'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:17.442759 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-80f26b01-5bbe-433a-942d-db3bbb8f8893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '858b9c3296554422855b97ca1f8629c0', 'request_id': 'req-80f26b01-5bbe-433a-942d-db3bbb8f8893'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:17.454251 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-80f26b01-5bbe-433a-942d-db3bbb8f8893 admin admin] The token cache is being invalidate because project 858b9c3296554422855b97ca1f8629c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:17.454537 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-80f26b01-5bbe-433a-942d-db3bbb8f8893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:17.455232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-80f26b01-5bbe-433a-942d-db3bbb8f8893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:17.456758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1314/2592] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:16 2022] DELETE /identity/v3/projects/858b9c3296554422855b97ca1f8629c0 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:17.461808 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9678627-8b3a-4128-bc3e-8f86aca8342d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:17.462086 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9678627-8b3a-4128-bc3e-8f86aca8342d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:17.462358 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9678627-8b3a-4128-bc3e-8f86aca8342d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:17.484444 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a9678627-8b3a-4128-bc3e-8f86aca8342d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:17.485687 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a9678627-8b3a-4128-bc3e-8f86aca8342d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:17.504151 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-12ee9efa-0405-4ddb-84aa-a57f07816824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32bf0639aeca4c649b5255a43970846b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:17.552936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1315/2593] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:17.562851 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462696b-72e2-446a-b582-db41086df8a1 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:17.563186 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462696b-72e2-446a-b582-db41086df8a1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:17.563459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462696b-72e2-446a-b582-db41086df8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:17.573912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-12ee9efa-0405-4ddb-84aa-a57f07816824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32bf0639aeca4c649b5255a43970846b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:17.584042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3462696b-72e2-446a-b582-db41086df8a1 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:17.585424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3462696b-72e2-446a-b582-db41086df8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:17.588336 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-12ee9efa-0405-4ddb-84aa-a57f07816824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32bf0639aeca4c649b5255a43970846b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:17.589052 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-12ee9efa-0405-4ddb-84aa-a57f07816824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32bf0639aeca4c649b5255a43970846b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:17.595052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1316/2594] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:17.598048 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462696b-72e2-446a-b582-db41086df8a1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:17.600905 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-12ee9efa-0405-4ddb-84aa-a57f07816824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32bf0639aeca4c649b5255a43970846b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:17.607727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1279/2595] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:17 2022] DELETE /identity/v3/users/32bf0639aeca4c649b5255a43970846b => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:17.608983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462696b-72e2-446a-b582-db41086df8a1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:17.674793 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eadbfd-a2f7-4a48-829c-532265646e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:17.675372 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eadbfd-a2f7-4a48-829c-532265646e79 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:17.675798 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eadbfd-a2f7-4a48-829c-532265646e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:17.676082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eadbfd-a2f7-4a48-829c-532265646e79 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:17.676755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eadbfd-a2f7-4a48-829c-532265646e79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:08:17.678472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eadbfd-a2f7-4a48-829c-532265646e79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:17.680199 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eadbfd-a2f7-4a48-829c-532265646e79 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:17.697651 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1317/2596] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:08:17.699836 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee9efa-0405-4ddb-84aa-a57f07816824 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:17.711223 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee9efa-0405-4ddb-84aa-a57f07816824 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:17.715964 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d95eb-bb2a-4123-8c75-d735075e853e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:17.716568 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d95eb-bb2a-4123-8c75-d735075e853e admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:17.716839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d95eb-bb2a-4123-8c75-d735075e853e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:17.717105 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d95eb-bb2a-4123-8c75-d735075e853e admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:17.717746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d95eb-bb2a-4123-8c75-d735075e853e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:17.719696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d95eb-bb2a-4123-8c75-d735075e853e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:17.820370 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1280/2597] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:17 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:17.824681 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eadbfd-a2f7-4a48-829c-532265646e79 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:17.840432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eadbfd-a2f7-4a48-829c-532265646e79 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:17.883792 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:17.910004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28227db9-5f23-455a-8dc2-a7792e86364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:17.910644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28227db9-5f23-455a-8dc2-a7792e86364d admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:17.910953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28227db9-5f23-455a-8dc2-a7792e86364d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:17.911238 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28227db9-5f23-455a-8dc2-a7792e86364d admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:17.911909 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28227db9-5f23-455a-8dc2-a7792e86364d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:17.913893 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28227db9-5f23-455a-8dc2-a7792e86364d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:18.021361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1318/2598] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:17 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:18.025764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d95eb-bb2a-4123-8c75-d735075e853e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:18.036359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d95eb-bb2a-4123-8c75-d735075e853e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:18.040899 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893daff-55a1-4d1a-8d0a-6e1a7ffdcc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:18.041492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893daff-55a1-4d1a-8d0a-6e1a7ffdcc26 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:18.041770 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893daff-55a1-4d1a-8d0a-6e1a7ffdcc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:18.042044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893daff-55a1-4d1a-8d0a-6e1a7ffdcc26 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:18.042674 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893daff-55a1-4d1a-8d0a-6e1a7ffdcc26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:18.044933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893daff-55a1-4d1a-8d0a-6e1a7ffdcc26 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:18.056002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1281/2599] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:18.060294 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28227db9-5f23-455a-8dc2-a7792e86364d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:18.070846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893daff-55a1-4d1a-8d0a-6e1a7ffdcc26 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:18.071405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28227db9-5f23-455a-8dc2-a7792e86364d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:18.073086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5893daff-55a1-4d1a-8d0a-6e1a7ffdcc26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:18.076675 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28891311-2cf7-43a9-bedf-c3b727f70217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:18.077194 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28891311-2cf7-43a9-bedf-c3b727f70217 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:18.077594 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28891311-2cf7-43a9-bedf-c3b727f70217 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:18.077991 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28891311-2cf7-43a9-bedf-c3b727f70217 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:18.079140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28891311-2cf7-43a9-bedf-c3b727f70217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:18.080723 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28891311-2cf7-43a9-bedf-c3b727f70217 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:18.093122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893daff-55a1-4d1a-8d0a-6e1a7ffdcc26 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:18.094156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1319/2600] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:18.098407 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28891311-2cf7-43a9-bedf-c3b727f70217 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:18.099361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0150680-fc6b-4ee4-b718-7e915a8ca4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:18.100136 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0150680-fc6b-4ee4-b718-7e915a8ca4de admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:18.100415 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0150680-fc6b-4ee4-b718-7e915a8ca4de admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:18.100695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0150680-fc6b-4ee4-b718-7e915a8ca4de admin admin] PATH_INFO: `/v3/projects/9749a26f541c4e8faed611aff675972d` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:18.101826 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0150680-fc6b-4ee4-b718-7e915a8ca4de admin admin] RBAC: Authorizing `identity:delete_project(project_id=9749a26f541c4e8faed611aff675972d)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:18.103822 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0150680-fc6b-4ee4-b718-7e915a8ca4de admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:18.110488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28891311-2cf7-43a9-bedf-c3b727f70217 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:18.115725 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f798d58-eb8b-4550-a6e5-c101c2b2e110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:18.116378 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f798d58-eb8b-4550-a6e5-c101c2b2e110 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:18.116660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f798d58-eb8b-4550-a6e5-c101c2b2e110 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:18.116933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f798d58-eb8b-4550-a6e5-c101c2b2e110 admin admin] PATH_INFO: `/v3/projects/87d3cfd3bc3a401c981d93b80b7cd6e1/users/5c096597060e4b25bab7d546979a21c6/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:18.132748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f798d58-eb8b-4550-a6e5-c101c2b2e110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d3cfd3bc3a401c981d93b80b7cd6e1, user_id=5c096597060e4b25bab7d546979a21c6, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:18.135453 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f798d58-eb8b-4550-a6e5-c101c2b2e110 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:18.239467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1320/2601] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:18 2022] PUT /identity/v3/projects/87d3cfd3bc3a401c981d93b80b7cd6e1/users/5c096597060e4b25bab7d546979a21c6/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:18.251135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384d532-d239-4d7d-800c-bbf3055bbe8c None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:18.251424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384d532-d239-4d7d-800c-bbf3055bbe8c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:18.251697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384d532-d239-4d7d-800c-bbf3055bbe8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:18.254143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b0150680-fc6b-4ee4-b718-7e915a8ca4de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9749a26f541c4e8faed611aff675972d', 'request_id': 'req-b0150680-fc6b-4ee4-b718-7e915a8ca4de'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:18.260223 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b0150680-fc6b-4ee4-b718-7e915a8ca4de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9749a26f541c4e8faed611aff675972d', 'request_id': 'req-b0150680-fc6b-4ee4-b718-7e915a8ca4de'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:18.267417 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b0150680-fc6b-4ee4-b718-7e915a8ca4de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9749a26f541c4e8faed611aff675972d', 'request_id': 'req-b0150680-fc6b-4ee4-b718-7e915a8ca4de'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:18.270375 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:18.300213 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f384d532-d239-4d7d-800c-bbf3055bbe8c None None] MFA Rules not processed for user `5c096597060e4b25bab7d546979a21c6`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:18.301530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f384d532-d239-4d7d-800c-bbf3055bbe8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:18.380665 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b0150680-fc6b-4ee4-b718-7e915a8ca4de admin admin] The token cache is being invalidate because project 9749a26f541c4e8faed611aff675972d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:18.381078 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b0150680-fc6b-4ee4-b718-7e915a8ca4de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:18.381789 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b0150680-fc6b-4ee4-b718-7e915a8ca4de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:18.383355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1282/2602] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:18 2022] DELETE /identity/v3/projects/9749a26f541c4e8faed611aff675972d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:18.386756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1321/2603] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:18.388592 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b932f1-7432-49ce-aadc-4f028c1ce3b4 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:18.388875 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b932f1-7432-49ce-aadc-4f028c1ce3b4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:18.389001 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384d532-d239-4d7d-800c-bbf3055bbe8c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:18.389148 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b932f1-7432-49ce-aadc-4f028c1ce3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:18.409416 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-04b932f1-7432-49ce-aadc-4f028c1ce3b4 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:18.410619 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-04b932f1-7432-49ce-aadc-4f028c1ce3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:18.452201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384d532-d239-4d7d-800c-bbf3055bbe8c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:18.456839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faff95e-036e-498b-bfa6-93012326b9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:18.457444 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faff95e-036e-498b-bfa6-93012326b9ff admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:18.457726 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faff95e-036e-498b-bfa6-93012326b9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:18.458002 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faff95e-036e-498b-bfa6-93012326b9ff admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:18.458632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faff95e-036e-498b-bfa6-93012326b9ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:18.460559 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faff95e-036e-498b-bfa6-93012326b9ff admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:18.485389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1283/2604] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:18.499293 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ee8a1-ccf5-490e-95db-c1a59d45168a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:18.499754 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ee8a1-ccf5-490e-95db-c1a59d45168a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:18.500116 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ee8a1-ccf5-490e-95db-c1a59d45168a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:18.519949 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b30ee8a1-ccf5-490e-95db-c1a59d45168a None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:18.521140 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b30ee8a1-ccf5-490e-95db-c1a59d45168a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:18.530127 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1284/2605] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:18.534376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ee8a1-ccf5-490e-95db-c1a59d45168a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:18.543924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ee8a1-ccf5-490e-95db-c1a59d45168a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:18.549457 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1322/2606] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:18 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:18.551161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff19ea-a698-47d3-8fed-5039e7707496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:18.551714 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff19ea-a698-47d3-8fed-5039e7707496 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:18.551990 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff19ea-a698-47d3-8fed-5039e7707496 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:18.552260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff19ea-a698-47d3-8fed-5039e7707496 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:18.552913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff19ea-a698-47d3-8fed-5039e7707496 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:08:18.553618 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff19ea-a698-47d3-8fed-5039e7707496 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:18.554532 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faff95e-036e-498b-bfa6-93012326b9ff admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:18.556294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff19ea-a698-47d3-8fed-5039e7707496 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:18.560501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4faff95e-036e-498b-bfa6-93012326b9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:18.572309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:18.573718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1285/2607] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:08:18.578218 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff19ea-a698-47d3-8fed-5039e7707496 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:18.589275 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff19ea-a698-47d3-8fed-5039e7707496 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:18.663947 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1667f7ad-6389-433a-a12e-1e3280c19ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:18.664787 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667f7ad-6389-433a-a12e-1e3280c19ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:18.665246 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667f7ad-6389-433a-a12e-1e3280c19ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:18.665687 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667f7ad-6389-433a-a12e-1e3280c19ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:18.666605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1667f7ad-6389-433a-a12e-1e3280c19ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:18.668605 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1667f7ad-6389-433a-a12e-1e3280c19ed5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:18.672416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faff95e-036e-498b-bfa6-93012326b9ff admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:18.677228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2cc7a-1ea0-438f-a671-177de2fcfcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:18.677862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2cc7a-1ea0-438f-a671-177de2fcfcb2 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:18.678137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2cc7a-1ea0-438f-a671-177de2fcfcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:18.678410 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2cc7a-1ea0-438f-a671-177de2fcfcb2 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:18.679061 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2cc7a-1ea0-438f-a671-177de2fcfcb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:18.680967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2cc7a-1ea0-438f-a671-177de2fcfcb2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:18.779932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1286/2608] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:18 2022] POST /identity/v3/projects => generated 379 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:18.783842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1667f7ad-6389-433a-a12e-1e3280c19ed5 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:18.808700 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1323/2609] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:18 2022] POST /identity/v3/users => generated 300 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:18.817738 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2cc7a-1ea0-438f-a671-177de2fcfcb2 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:18.828467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2cc7a-1ea0-438f-a671-177de2fcfcb2 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:18.849299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1667f7ad-6389-433a-a12e-1e3280c19ed5 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:18.854782 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330f490-b667-4f08-867b-a206535d1719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:18.855406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330f490-b667-4f08-867b-a206535d1719 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:18.855681 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330f490-b667-4f08-867b-a206535d1719 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:18.855958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330f490-b667-4f08-867b-a206535d1719 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:18.857494 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5330f490-b667-4f08-867b-a206535d1719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:18.859116 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5330f490-b667-4f08-867b-a206535d1719 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:18.867409 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:18.887834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a63c3b-63ec-42bf-9e4e-330a55795b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:18.888468 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a63c3b-63ec-42bf-9e4e-330a55795b7a admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:18.888740 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a63c3b-63ec-42bf-9e4e-330a55795b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:18.889006 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a63c3b-63ec-42bf-9e4e-330a55795b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:18.889634 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a63c3b-63ec-42bf-9e4e-330a55795b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:18.891262 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a63c3b-63ec-42bf-9e4e-330a55795b7a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:18.901302 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1324/2610] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:18.905477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a63c3b-63ec-42bf-9e4e-330a55795b7a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:18.921478 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a63c3b-63ec-42bf-9e4e-330a55795b7a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:18.921478 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f17275-1dbc-440b-a91c-3dece4137bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:18.921478 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f17275-1dbc-440b-a91c-3dece4137bea admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:18.921478 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f17275-1dbc-440b-a91c-3dece4137bea admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:18.921844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f17275-1dbc-440b-a91c-3dece4137bea admin admin] PATH_INFO: `/v3/projects/936c0ba3487c47f89b246f397502e857/users/e7d631e66d814c8aa6349058e8f8f44a/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:18.937343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f17275-1dbc-440b-a91c-3dece4137bea admin admin] RBAC: Authorizing `identity:create_grant(project_id=936c0ba3487c47f89b246f397502e857, user_id=e7d631e66d814c8aa6349058e8f8f44a, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:18.940018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f17275-1dbc-440b-a91c-3dece4137bea admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:18.983521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1287/2611] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:18 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:18.987815 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330f490-b667-4f08-867b-a206535d1719 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:18.998805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330f490-b667-4f08-867b-a206535d1719 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:19.003476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a987d5-270e-410f-8d34-125f25e858a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:19.004062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a987d5-270e-410f-8d34-125f25e858a5 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:19.004342 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a987d5-270e-410f-8d34-125f25e858a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:19.004616 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a987d5-270e-410f-8d34-125f25e858a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:19.005277 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a987d5-270e-410f-8d34-125f25e858a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:19.006886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a987d5-270e-410f-8d34-125f25e858a5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:19.016911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1288/2612] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:19.024105 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a987d5-270e-410f-8d34-125f25e858a5 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:19.033795 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a987d5-270e-410f-8d34-125f25e858a5 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:19.038304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac590fb-ab25-4066-8ee6-6c31082593fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:19.038876 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac590fb-ab25-4066-8ee6-6c31082593fa admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:19.039180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac590fb-ab25-4066-8ee6-6c31082593fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:19.039446 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac590fb-ab25-4066-8ee6-6c31082593fa admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:19.040063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac590fb-ab25-4066-8ee6-6c31082593fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:19.041640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac590fb-ab25-4066-8ee6-6c31082593fa admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:19.053433 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1289/2613] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:19.058352 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac590fb-ab25-4066-8ee6-6c31082593fa admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:19.068764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac590fb-ab25-4066-8ee6-6c31082593fa admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:19.073550 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9149e4f-3cc7-479c-87e3-8925e464530c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:19.074450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9149e4f-3cc7-479c-87e3-8925e464530c admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:19.075463 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9149e4f-3cc7-479c-87e3-8925e464530c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:19.075740 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9149e4f-3cc7-479c-87e3-8925e464530c admin admin] PATH_INFO: `/v3/projects/48c539bd128e4637ba60f521f7dbba9b/users/a0bb160c9ea3458a9345e3c3a9186079/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:19.080615 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1325/2614] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:18 2022] PUT /identity/v3/projects/936c0ba3487c47f89b246f397502e857/users/e7d631e66d814c8aa6349058e8f8f44a/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:19.088237 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f17275-1dbc-440b-a91c-3dece4137bea admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:19.094700 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9149e4f-3cc7-479c-87e3-8925e464530c admin admin] RBAC: Authorizing `identity:create_grant(project_id=48c539bd128e4637ba60f521f7dbba9b, user_id=a0bb160c9ea3458a9345e3c3a9186079, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:19.097642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9149e4f-3cc7-479c-87e3-8925e464530c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:19.151423 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f17275-1dbc-440b-a91c-3dece4137bea admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:19.159448 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1290/2615] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:19 2022] PUT /identity/v3/projects/48c539bd128e4637ba60f521f7dbba9b/users/a0bb160c9ea3458a9345e3c3a9186079/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:19.169992 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:19.170695 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cb58b-11a7-4e8b-9a24-0e1c202b129c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:19.170998 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cb58b-11a7-4e8b-9a24-0e1c202b129c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:19.171331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cb58b-11a7-4e8b-9a24-0e1c202b129c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:19.203010 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e93cb58b-11a7-4e8b-9a24-0e1c202b129c None None] MFA Rules not processed for user `a0bb160c9ea3458a9345e3c3a9186079`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:19.204242 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e93cb58b-11a7-4e8b-9a24-0e1c202b129c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:19.225326 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfc99d-2b19-45cf-938d-833d5811a75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:19.225926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfc99d-2b19-45cf-938d-833d5811a75d admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:19.226228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfc99d-2b19-45cf-938d-833d5811a75d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:19.226495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dfc99d-2b19-45cf-938d-833d5811a75d admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:19.227208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dfc99d-2b19-45cf-938d-833d5811a75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:19.228834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dfc99d-2b19-45cf-938d-833d5811a75d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:19.242423 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1326/2616] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:19.246395 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfc99d-2b19-45cf-938d-833d5811a75d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:19.254197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dfc99d-2b19-45cf-938d-833d5811a75d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:19.259582 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d4720-5e4d-4e52-9a6d-e0367442c9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:19.260232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d4720-5e4d-4e52-9a6d-e0367442c9ae admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:19.260579 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d4720-5e4d-4e52-9a6d-e0367442c9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:19.260780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d4720-5e4d-4e52-9a6d-e0367442c9ae admin admin] PATH_INFO: `/v3/projects/936c0ba3487c47f89b246f397502e857/users/e7d631e66d814c8aa6349058e8f8f44a/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:19.262709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d4720-5e4d-4e52-9a6d-e0367442c9ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=936c0ba3487c47f89b246f397502e857, user_id=e7d631e66d814c8aa6349058e8f8f44a, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:19.266413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d4720-5e4d-4e52-9a6d-e0367442c9ae admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:19.279820 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1327/2617] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:19 2022] PUT /identity/v3/projects/936c0ba3487c47f89b246f397502e857/users/e7d631e66d814c8aa6349058e8f8f44a/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:19.288989 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1291/2618] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:19.293303 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d4720-5e4d-4e52-9a6d-e0367442c9ae admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:19.293630 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c8996-5628-47a4-9629-1b8de133d593 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:19.294480 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c8996-5628-47a4-9629-1b8de133d593 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:19.294755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c8996-5628-47a4-9629-1b8de133d593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:19.331135 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d81c8996-5628-47a4-9629-1b8de133d593 None None] MFA Rules not processed for user `e7d631e66d814c8aa6349058e8f8f44a`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:19.332160 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d81c8996-5628-47a4-9629-1b8de133d593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:19.369228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d4720-5e4d-4e52-9a6d-e0367442c9ae admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:19.376757 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb12eb-6dbb-43b2-9ae6-fa79582b4115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:19.376757 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb12eb-6dbb-43b2-9ae6-fa79582b4115 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:19.376757 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb12eb-6dbb-43b2-9ae6-fa79582b4115 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:19.376757 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb12eb-6dbb-43b2-9ae6-fa79582b4115 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:19.377363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb12eb-6dbb-43b2-9ae6-fa79582b4115 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:19.380004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb12eb-6dbb-43b2-9ae6-fa79582b4115 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:19.415193 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1292/2619] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:19.426778 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed81d0-c17b-4a76-a0c1-47bfb1d9e5c6 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:19.427136 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed81d0-c17b-4a76-a0c1-47bfb1d9e5c6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:19.427416 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed81d0-c17b-4a76-a0c1-47bfb1d9e5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:19.447161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f1ed81d0-c17b-4a76-a0c1-47bfb1d9e5c6 None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:19.448354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f1ed81d0-c17b-4a76-a0c1-47bfb1d9e5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:19.460788 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:19.495833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1328/2620] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:19 2022] POST /identity/v3/projects => generated 379 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:19.495833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb12eb-6dbb-43b2-9ae6-fa79582b4115 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:19.500618 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-57bb12eb-6dbb-43b2-9ae6-fa79582b4115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:19.513958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1293/2621] 10.222.0.91 () {58 vars in 1009 bytes} [Thu Sep 8 14:08:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:19.514825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed81d0-c17b-4a76-a0c1-47bfb1d9e5c6 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:19.529711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed81d0-c17b-4a76-a0c1-47bfb1d9e5c6 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:19.562429 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb12eb-6dbb-43b2-9ae6-fa79582b4115 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:19.566287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755d235-88e5-4e41-a013-e553ba1c4a80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:19.567003 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755d235-88e5-4e41-a013-e553ba1c4a80 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:19.568632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755d235-88e5-4e41-a013-e553ba1c4a80 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:19.568632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755d235-88e5-4e41-a013-e553ba1c4a80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:19.593076 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fb216-f53a-4054-9e9b-ce68e9a47b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:19.593681 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fb216-f53a-4054-9e9b-ce68e9a47b37 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:19.593955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fb216-f53a-4054-9e9b-ce68e9a47b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:19.594222 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fb216-f53a-4054-9e9b-ce68e9a47b37 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:19.594846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57fb216-f53a-4054-9e9b-ce68e9a47b37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:19.596771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57fb216-f53a-4054-9e9b-ce68e9a47b37 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:19.605443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:19.633426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755d235-88e5-4e41-a013-e553ba1c4a80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:19.634847 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755d235-88e5-4e41-a013-e553ba1c4a80 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:19.645299 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1329/2622] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:19.646134 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755d235-88e5-4e41-a013-e553ba1c4a80 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:19.647696 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2755d235-88e5-4e41-a013-e553ba1c4a80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:19.702930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1294/2623] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:19 2022] POST /identity/v3/users => generated 290 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:19.704727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fb216-f53a-4054-9e9b-ce68e9a47b37 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:19.714758 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fb216-f53a-4054-9e9b-ce68e9a47b37 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:19.718251 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755d235-88e5-4e41-a013-e553ba1c4a80 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:19.720817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb751787-ae74-417a-84b6-07fac51f4d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:19.720817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb751787-ae74-417a-84b6-07fac51f4d2b admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:19.722017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb751787-ae74-417a-84b6-07fac51f4d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:19.722017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb751787-ae74-417a-84b6-07fac51f4d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:19.722017 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb751787-ae74-417a-84b6-07fac51f4d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:19.723504 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7c47c-cb30-4ab7-a19d-4b691c1f79a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:19.723637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb751787-ae74-417a-84b6-07fac51f4d2b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:19.724100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7c47c-cb30-4ab7-a19d-4b691c1f79a4 service swift] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:19.724401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7c47c-cb30-4ab7-a19d-4b691c1f79a4 service swift] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:19.724669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7c47c-cb30-4ab7-a19d-4b691c1f79a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:19.741191 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1295/2624] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:19.747989 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb751787-ae74-417a-84b6-07fac51f4d2b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:19.758107 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb751787-ae74-417a-84b6-07fac51f4d2b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:19.764206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d6a55-27a1-4c7f-a66f-34715c93345a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:19.764814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d6a55-27a1-4c7f-a66f-34715c93345a admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:19.765088 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d6a55-27a1-4c7f-a66f-34715c93345a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:19.765373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d6a55-27a1-4c7f-a66f-34715c93345a admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:19.766008 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d6a55-27a1-4c7f-a66f-34715c93345a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:19.768321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d6a55-27a1-4c7f-a66f-34715c93345a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:19.782092 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1296/2625] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:19.786084 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d6a55-27a1-4c7f-a66f-34715c93345a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:19.788449 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7c47c-cb30-4ab7-a19d-4b691c1f79a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:19.789958 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7c47c-cb30-4ab7-a19d-4b691c1f79a4 service swift] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:19.796734 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d6a55-27a1-4c7f-a66f-34715c93345a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:19.798267 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1330/2626] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 14:08:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:08:19.801472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b53ae0-df69-427d-90c8-f663109db743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:19.802214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b53ae0-df69-427d-90c8-f663109db743 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:19.802531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b53ae0-df69-427d-90c8-f663109db743 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:19.802750 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b53ae0-df69-427d-90c8-f663109db743 admin admin] PATH_INFO: `/v3/projects/d0fa837cefcf44f1a5fc12dd19756991/users/04af9d0b99f54f7a9951617ccca812fa/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:19.818114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b53ae0-df69-427d-90c8-f663109db743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0fa837cefcf44f1a5fc12dd19756991, user_id=04af9d0b99f54f7a9951617ccca812fa, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:19.821010 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b53ae0-df69-427d-90c8-f663109db743 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:19.905816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1297/2627] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:19 2022] PUT /identity/v3/projects/d0fa837cefcf44f1a5fc12dd19756991/users/04af9d0b99f54f7a9951617ccca812fa/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:19.916453 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35bdd-87ad-44cc-8509-7cf6b9dd4828 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:19.916776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35bdd-87ad-44cc-8509-7cf6b9dd4828 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:19.916993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35bdd-87ad-44cc-8509-7cf6b9dd4828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:19.953054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-5aa35bdd-87ad-44cc-8509-7cf6b9dd4828 None None] MFA Rules not processed for user `04af9d0b99f54f7a9951617ccca812fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:19.954382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5aa35bdd-87ad-44cc-8509-7cf6b9dd4828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:20.047784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1331/2628] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:20.052155 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b53ae0-df69-427d-90c8-f663109db743 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:20.127091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b53ae0-df69-427d-90c8-f663109db743 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:20.132931 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f5482-f06b-4d1e-8f21-0dd115322bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:20.134158 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f5482-f06b-4d1e-8f21-0dd115322bdd admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:20.134435 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f5482-f06b-4d1e-8f21-0dd115322bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:20.134717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f5482-f06b-4d1e-8f21-0dd115322bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:20.135509 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f5482-f06b-4d1e-8f21-0dd115322bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:20.137753 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f5482-f06b-4d1e-8f21-0dd115322bdd admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:20.143305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:20.276696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1298/2629] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:20 2022] POST /identity/v3/projects => generated 379 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:20.278090 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa35bdd-87ad-44cc-8509-7cf6b9dd4828 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:20.286688 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:20.290998 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa35bdd-87ad-44cc-8509-7cf6b9dd4828 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:20.354143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c943445-70da-467d-b488-f8646b3a70ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:20.354760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c943445-70da-467d-b488-f8646b3a70ba admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:20.355076 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c943445-70da-467d-b488-f8646b3a70ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:20.355362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c943445-70da-467d-b488-f8646b3a70ba admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:20.356002 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c943445-70da-467d-b488-f8646b3a70ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:20.358722 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c943445-70da-467d-b488-f8646b3a70ba admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:20.456258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1332/2630] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:20 2022] POST /identity/v3/users => generated 296 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:20.460479 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f5482-f06b-4d1e-8f21-0dd115322bdd admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:20.472496 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f5482-f06b-4d1e-8f21-0dd115322bdd admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:20.493406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dc9bd-7598-442b-8993-103ba02b1771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:20.494409 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dc9bd-7598-442b-8993-103ba02b1771 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:20.494696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dc9bd-7598-442b-8993-103ba02b1771 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:20.495268 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dc9bd-7598-442b-8993-103ba02b1771 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:20.496235 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7dc9bd-7598-442b-8993-103ba02b1771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:20.497890 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7dc9bd-7598-442b-8993-103ba02b1771 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:20.525863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1299/2631] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:20.530975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c943445-70da-467d-b488-f8646b3a70ba admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:20.552000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c943445-70da-467d-b488-f8646b3a70ba admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:20.567882 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b815bf-f943-4071-8053-0ae7760decef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:20.573124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b815bf-f943-4071-8053-0ae7760decef admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:20.573124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b815bf-f943-4071-8053-0ae7760decef admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:20.573124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b815bf-f943-4071-8053-0ae7760decef admin admin] PATH_INFO: `/v3/projects/b845777a282e46ff8c5a95d7aa967ec5/users/f2324a64a858460ca6fa6c0e6f6f0461/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:20.611652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b815bf-f943-4071-8053-0ae7760decef admin admin] RBAC: Authorizing `identity:create_grant(project_id=b845777a282e46ff8c5a95d7aa967ec5, user_id=f2324a64a858460ca6fa6c0e6f6f0461, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:20.614618 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b815bf-f943-4071-8053-0ae7760decef admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:20.694703 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1333/2632] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:20 2022] PUT /identity/v3/projects/b845777a282e46ff8c5a95d7aa967ec5/users/f2324a64a858460ca6fa6c0e6f6f0461/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:20.697376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dc9bd-7598-442b-8993-103ba02b1771 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:20.710669 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dc9bd-7598-442b-8993-103ba02b1771 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:20.822937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e4e9a-7cb1-4ec4-b810-e50881304e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:20.823760 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e4e9a-7cb1-4ec4-b810-e50881304e21 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:20.824097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e4e9a-7cb1-4ec4-b810-e50881304e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:20.824455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e4e9a-7cb1-4ec4-b810-e50881304e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:20.825314 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e4e9a-7cb1-4ec4-b810-e50881304e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:20.827321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688e4e9a-7cb1-4ec4-b810-e50881304e21 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:20.838195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1300/2633] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:20.844487 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b815bf-f943-4071-8053-0ae7760decef admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:20.860889 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b815bf-f943-4071-8053-0ae7760decef admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:20.866798 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b4ee7-612a-4f8a-861f-909a95137770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:20.867686 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b4ee7-612a-4f8a-861f-909a95137770 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:20.868034 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b4ee7-612a-4f8a-861f-909a95137770 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:20.868384 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b4ee7-612a-4f8a-861f-909a95137770 admin admin] PATH_INFO: `/v3/projects/b845777a282e46ff8c5a95d7aa967ec5/users/f2324a64a858460ca6fa6c0e6f6f0461/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:20.870661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b4ee7-612a-4f8a-861f-909a95137770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b845777a282e46ff8c5a95d7aa967ec5, user_id=f2324a64a858460ca6fa6c0e6f6f0461, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:20.874511 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b4ee7-612a-4f8a-861f-909a95137770 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:20.889837 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1334/2634] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:20 2022] PUT /identity/v3/projects/b845777a282e46ff8c5a95d7aa967ec5/users/f2324a64a858460ca6fa6c0e6f6f0461/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:20.901499 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45117a5f-bb84-40e1-8b2b-7a73207b91fa None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:20.902123 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45117a5f-bb84-40e1-8b2b-7a73207b91fa None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:20.902625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45117a5f-bb84-40e1-8b2b-7a73207b91fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:20.966501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-45117a5f-bb84-40e1-8b2b-7a73207b91fa None None] MFA Rules not processed for user `f2324a64a858460ca6fa6c0e6f6f0461`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:20.967985 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-45117a5f-bb84-40e1-8b2b-7a73207b91fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:20.993803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b4ee7-612a-4f8a-861f-909a95137770 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:21.086264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b4ee7-612a-4f8a-861f-909a95137770 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:21.092391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affc8d5-4c62-4708-830b-3be74ba5d85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:21.093267 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affc8d5-4c62-4708-830b-3be74ba5d85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:21.093702 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affc8d5-4c62-4708-830b-3be74ba5d85b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:21.094274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affc8d5-4c62-4708-830b-3be74ba5d85b admin admin] PATH_INFO: `/v3/users/5c096597060e4b25bab7d546979a21c6` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:21.095714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affc8d5-4c62-4708-830b-3be74ba5d85b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c096597060e4b25bab7d546979a21c6)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:21.097797 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affc8d5-4c62-4708-830b-3be74ba5d85b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:21.143362 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1301/2635] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:21.152096 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45117a5f-bb84-40e1-8b2b-7a73207b91fa None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:21.248897 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45117a5f-bb84-40e1-8b2b-7a73207b91fa None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:21.253351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696919f3-80c3-4f90-875a-92002483ae8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:21.253965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696919f3-80c3-4f90-875a-92002483ae8f service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:21.254385 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696919f3-80c3-4f90-875a-92002483ae8f service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:21.254842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696919f3-80c3-4f90-875a-92002483ae8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:21.268195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696919f3-80c3-4f90-875a-92002483ae8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:21.270800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696919f3-80c3-4f90-875a-92002483ae8f service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:21.278201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:21.286416 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1302/2636] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:21 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:21.503368 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6affc8d5-4c62-4708-830b-3be74ba5d85b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c096597060e4b25bab7d546979a21c6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:21.553749 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696919f3-80c3-4f90-875a-92002483ae8f service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:21.555744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-696919f3-80c3-4f90-875a-92002483ae8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:21.580475 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696919f3-80c3-4f90-875a-92002483ae8f service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:21.582876 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6affc8d5-4c62-4708-830b-3be74ba5d85b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c096597060e4b25bab7d546979a21c6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:21.583656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6affc8d5-4c62-4708-830b-3be74ba5d85b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c096597060e4b25bab7d546979a21c6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:21.586879 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:21.594702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10726ce-2f27-4492-9078-07597173d976 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:21.596093 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10726ce-2f27-4492-9078-07597173d976 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:21.596564 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10726ce-2f27-4492-9078-07597173d976 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:21.597119 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10726ce-2f27-4492-9078-07597173d976 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:21.597637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6affc8d5-4c62-4708-830b-3be74ba5d85b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c096597060e4b25bab7d546979a21c6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:21.599867 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b10726ce-2f27-4492-9078-07597173d976 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:21.620457 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6affc8d5-4c62-4708-830b-3be74ba5d85b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c096597060e4b25bab7d546979a21c6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:21.633049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1335/2637] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:20 2022] DELETE /identity/v3/users/5c096597060e4b25bab7d546979a21c6 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:21.638709 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9b6ae-a2ca-4294-b376-53859c774d9f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:21.639021 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9b6ae-a2ca-4294-b376-53859c774d9f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:21.640207 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9b6ae-a2ca-4294-b376-53859c774d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:21.696159 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3fe9b6ae-a2ca-4294-b376-53859c774d9f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:21.697356 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3fe9b6ae-a2ca-4294-b376-53859c774d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:21.748926 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10726ce-2f27-4492-9078-07597173d976 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:21.750775 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10726ce-2f27-4492-9078-07597173d976 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:21.766635 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1303/2638] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:08:21.824774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1336/2639] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:21.838040 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10726ce-2f27-4492-9078-07597173d976 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:21.841904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b10726ce-2f27-4492-9078-07597173d976 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:21.921668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10726ce-2f27-4492-9078-07597173d976 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:22.009894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a30df2-2c08-4933-901c-fa1e211688ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:22.010647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a30df2-2c08-4933-901c-fa1e211688ae service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:22.011028 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a30df2-2c08-4933-901c-fa1e211688ae service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:22.011310 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a30df2-2c08-4933-901c-fa1e211688ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:22.033332 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a30df2-2c08-4933-901c-fa1e211688ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:22.034930 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a30df2-2c08-4933-901c-fa1e211688ae service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:22.047683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1304/2640] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:22.548686 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9b6ae-a2ca-4294-b376-53859c774d9f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:22.550396 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3fe9b6ae-a2ca-4294-b376-53859c774d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:22.573459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9b6ae-a2ca-4294-b376-53859c774d9f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:22.586766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24c75c-a632-4cd0-aea9-930e33d07b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:22.587432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24c75c-a632-4cd0-aea9-930e33d07b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:22.587713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24c75c-a632-4cd0-aea9-930e33d07b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:22.587985 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24c75c-a632-4cd0-aea9-930e33d07b90 admin admin] PATH_INFO: `/v3/projects/87d3cfd3bc3a401c981d93b80b7cd6e1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:22.589109 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24c75c-a632-4cd0-aea9-930e33d07b90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87d3cfd3bc3a401c981d93b80b7cd6e1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:22.592091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24c75c-a632-4cd0-aea9-930e33d07b90 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:22.738886 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8e24c75c-a632-4cd0-aea9-930e33d07b90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87d3cfd3bc3a401c981d93b80b7cd6e1', 'request_id': 'req-8e24c75c-a632-4cd0-aea9-930e33d07b90'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:22.817194 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8e24c75c-a632-4cd0-aea9-930e33d07b90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87d3cfd3bc3a401c981d93b80b7cd6e1', 'request_id': 'req-8e24c75c-a632-4cd0-aea9-930e33d07b90'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:22.825386 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8e24c75c-a632-4cd0-aea9-930e33d07b90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87d3cfd3bc3a401c981d93b80b7cd6e1', 'request_id': 'req-8e24c75c-a632-4cd0-aea9-930e33d07b90'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:22.841167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8e24c75c-a632-4cd0-aea9-930e33d07b90 admin admin] The token cache is being invalidate because project 87d3cfd3bc3a401c981d93b80b7cd6e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:22.841953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8e24c75c-a632-4cd0-aea9-930e33d07b90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:22.842947 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8e24c75c-a632-4cd0-aea9-930e33d07b90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:22.845192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1337/2641] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:22 2022] DELETE /identity/v3/projects/87d3cfd3bc3a401c981d93b80b7cd6e1 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:22.850034 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a30df2-2c08-4933-901c-fa1e211688ae service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:22.862618 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a30df2-2c08-4933-901c-fa1e211688ae service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:22.922811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:22.932202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3adddb-9279-408b-a7e3-1ace7772bf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:22.932923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3adddb-9279-408b-a7e3-1ace7772bf9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:22.933284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3adddb-9279-408b-a7e3-1ace7772bf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:22.933657 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3adddb-9279-408b-a7e3-1ace7772bf9d admin admin] PATH_INFO: `/v3/users/e7d631e66d814c8aa6349058e8f8f44a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:22.935459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3adddb-9279-408b-a7e3-1ace7772bf9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7d631e66d814c8aa6349058e8f8f44a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:22.939037 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3adddb-9279-408b-a7e3-1ace7772bf9d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:23.154787 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0f3adddb-9279-408b-a7e3-1ace7772bf9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7d631e66d814c8aa6349058e8f8f44a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:23.527954 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0f3adddb-9279-408b-a7e3-1ace7772bf9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7d631e66d814c8aa6349058e8f8f44a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:23.544157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0f3adddb-9279-408b-a7e3-1ace7772bf9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7d631e66d814c8aa6349058e8f8f44a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:23.544889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0f3adddb-9279-408b-a7e3-1ace7772bf9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7d631e66d814c8aa6349058e8f8f44a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:23.552570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0f3adddb-9279-408b-a7e3-1ace7772bf9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7d631e66d814c8aa6349058e8f8f44a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:23.559036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1305/2642] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:22 2022] DELETE /identity/v3/users/e7d631e66d814c8aa6349058e8f8f44a => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:24.132458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24c75c-a632-4cd0-aea9-930e33d07b90 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:24.134562 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8e24c75c-a632-4cd0-aea9-930e33d07b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:24.185991 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24c75c-a632-4cd0-aea9-930e33d07b90 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:24.255369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:24.266240 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ff621-d55b-47ae-9570-2ed14f6b1790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:24.267002 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ff621-d55b-47ae-9570-2ed14f6b1790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:24.267443 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ff621-d55b-47ae-9570-2ed14f6b1790 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:24.274352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ff621-d55b-47ae-9570-2ed14f6b1790 admin admin] PATH_INFO: `/v3/projects/936c0ba3487c47f89b246f397502e857` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:24.274352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ff621-d55b-47ae-9570-2ed14f6b1790 admin admin] RBAC: Authorizing `identity:delete_project(project_id=936c0ba3487c47f89b246f397502e857)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:24.274352 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ff621-d55b-47ae-9570-2ed14f6b1790 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:24.447211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d45ff621-d55b-47ae-9570-2ed14f6b1790 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '936c0ba3487c47f89b246f397502e857', 'request_id': 'req-d45ff621-d55b-47ae-9570-2ed14f6b1790'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:24.554348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d45ff621-d55b-47ae-9570-2ed14f6b1790 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '936c0ba3487c47f89b246f397502e857', 'request_id': 'req-d45ff621-d55b-47ae-9570-2ed14f6b1790'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:24.567216 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d45ff621-d55b-47ae-9570-2ed14f6b1790 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '936c0ba3487c47f89b246f397502e857', 'request_id': 'req-d45ff621-d55b-47ae-9570-2ed14f6b1790'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:24.590816 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d45ff621-d55b-47ae-9570-2ed14f6b1790 admin admin] The token cache is being invalidate because project 936c0ba3487c47f89b246f397502e857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:24.591392 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d45ff621-d55b-47ae-9570-2ed14f6b1790 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:24.592452 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d45ff621-d55b-47ae-9570-2ed14f6b1790 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:24.594386 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1338/2643] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:24 2022] DELETE /identity/v3/projects/936c0ba3487c47f89b246f397502e857 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:24.600093 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c0f7e-a3ba-4598-8a27-00e488e58295 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:24.600613 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c0f7e-a3ba-4598-8a27-00e488e58295 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:24.600980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c0f7e-a3ba-4598-8a27-00e488e58295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:24.626834 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-689c0f7e-a3ba-4598-8a27-00e488e58295 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:24.628353 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-689c0f7e-a3ba-4598-8a27-00e488e58295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:24.693901 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1306/2644] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:24.704580 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3d843-c7df-4e50-a9af-63e93aab4fd5 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:24.704932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3d843-c7df-4e50-a9af-63e93aab4fd5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:24.705323 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3d843-c7df-4e50-a9af-63e93aab4fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:24.735200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-81c3d843-c7df-4e50-a9af-63e93aab4fd5 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:24.736598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-81c3d843-c7df-4e50-a9af-63e93aab4fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:24.759740 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1339/2645] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:24.764312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c0f7e-a3ba-4598-8a27-00e488e58295 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:24.782996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c0f7e-a3ba-4598-8a27-00e488e58295 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:24.795355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df747efb-6353-4387-aca6-d39fe6b71d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:24.799961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df747efb-6353-4387-aca6-d39fe6b71d64 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:24.799961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df747efb-6353-4387-aca6-d39fe6b71d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:24.799961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df747efb-6353-4387-aca6-d39fe6b71d64 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:24.799961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df747efb-6353-4387-aca6-d39fe6b71d64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:08:24.799961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df747efb-6353-4387-aca6-d39fe6b71d64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:24.802027 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df747efb-6353-4387-aca6-d39fe6b71d64 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:24.821545 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1307/2646] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:08:24.823606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3d843-c7df-4e50-a9af-63e93aab4fd5 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:24.836169 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3d843-c7df-4e50-a9af-63e93aab4fd5 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:24.842432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e19af-935e-453e-bbea-6f85c2126a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:24.843083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e19af-935e-453e-bbea-6f85c2126a4c admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:24.843360 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e19af-935e-453e-bbea-6f85c2126a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:24.843629 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e19af-935e-453e-bbea-6f85c2126a4c admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:24.844261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e19af-935e-453e-bbea-6f85c2126a4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:24.846853 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e19af-935e-453e-bbea-6f85c2126a4c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:24.892250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df747efb-6353-4387-aca6-d39fe6b71d64 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:24.894140 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-df747efb-6353-4387-aca6-d39fe6b71d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:24.962210 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df747efb-6353-4387-aca6-d39fe6b71d64 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:24.965992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607c153-e8e6-4ef8-aa55-d7b35aabe074 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:24.966601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607c153-e8e6-4ef8-aa55-d7b35aabe074 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:24.966878 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607c153-e8e6-4ef8-aa55-d7b35aabe074 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:24.967175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607c153-e8e6-4ef8-aa55-d7b35aabe074 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:24.968931 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c607c153-e8e6-4ef8-aa55-d7b35aabe074 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:24.982539 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607c153-e8e6-4ef8-aa55-d7b35aabe074 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:24.984012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607c153-e8e6-4ef8-aa55-d7b35aabe074 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:24.995041 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1340/2647] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:24 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:25.000501 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e19af-935e-453e-bbea-6f85c2126a4c admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:25.005305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1308/2648] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:08:25.117936 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e19af-935e-453e-bbea-6f85c2126a4c admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:25.124242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dca7d-2149-495d-b700-a31ca9ddb1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:25.124992 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dca7d-2149-495d-b700-a31ca9ddb1cf admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:25.125336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dca7d-2149-495d-b700-a31ca9ddb1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:25.125686 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dca7d-2149-495d-b700-a31ca9ddb1cf admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:25.126605 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8dca7d-2149-495d-b700-a31ca9ddb1cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:25.128831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8dca7d-2149-495d-b700-a31ca9ddb1cf admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:25.232698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1341/2649] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:24 2022] POST /identity/v3/users => generated 287 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:25.237175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607c153-e8e6-4ef8-aa55-d7b35aabe074 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:25.244328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:25.249589 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607c153-e8e6-4ef8-aa55-d7b35aabe074 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:25.254478 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f3946-ecbe-4ba5-823a-a6f77562fd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:25.255177 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f3946-ecbe-4ba5-823a-a6f77562fd83 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:25.258153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f3946-ecbe-4ba5-823a-a6f77562fd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:25.258153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f3946-ecbe-4ba5-823a-a6f77562fd83 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:25.258153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f3946-ecbe-4ba5-823a-a6f77562fd83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:25.258153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f3946-ecbe-4ba5-823a-a6f77562fd83 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:25.272065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1309/2650] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:25.276333 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dca7d-2149-495d-b700-a31ca9ddb1cf admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:25.288927 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dca7d-2149-495d-b700-a31ca9ddb1cf admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:25.296120 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee626e-25ca-458c-a59c-f353a3279ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:25.296742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee626e-25ca-458c-a59c-f353a3279ead admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:25.297019 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee626e-25ca-458c-a59c-f353a3279ead admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:25.297497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee626e-25ca-458c-a59c-f353a3279ead admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:25.298149 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee626e-25ca-458c-a59c-f353a3279ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:25.299777 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee626e-25ca-458c-a59c-f353a3279ead admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:25.311972 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1342/2651] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:25.316811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f3946-ecbe-4ba5-823a-a6f77562fd83 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:25.330551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f3946-ecbe-4ba5-823a-a6f77562fd83 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:25.337805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab83c8-7e5a-4570-afa8-46bac4639cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:25.339643 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab83c8-7e5a-4570-afa8-46bac4639cd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:25.339643 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab83c8-7e5a-4570-afa8-46bac4639cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:25.339643 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab83c8-7e5a-4570-afa8-46bac4639cd7 admin admin] PATH_INFO: `/v3/projects/a1eaab11ad3e4ed1a282204cb6941ef2/users/d8eebf5044a64ff6b9b3d946ac2ab129/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:25.366721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab83c8-7e5a-4570-afa8-46bac4639cd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1eaab11ad3e4ed1a282204cb6941ef2, user_id=d8eebf5044a64ff6b9b3d946ac2ab129, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:25.369755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab83c8-7e5a-4570-afa8-46bac4639cd7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:25.465339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1310/2652] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:25 2022] PUT /identity/v3/projects/a1eaab11ad3e4ed1a282204cb6941ef2/users/d8eebf5044a64ff6b9b3d946ac2ab129/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:25.475646 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dce87-b885-4fb7-b76f-5d7dc3ed2909 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:25.476327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dce87-b885-4fb7-b76f-5d7dc3ed2909 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:25.476776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dce87-b885-4fb7-b76f-5d7dc3ed2909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:25.510365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-588dce87-b885-4fb7-b76f-5d7dc3ed2909 None None] MFA Rules not processed for user `d8eebf5044a64ff6b9b3d946ac2ab129`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:25.513100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-588dce87-b885-4fb7-b76f-5d7dc3ed2909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:25.546732 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:25.597362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1343/2653] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:25.602237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab83c8-7e5a-4570-afa8-46bac4639cd7 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:25.720480 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab83c8-7e5a-4570-afa8-46bac4639cd7 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:25.725637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167b3bb-ec04-49d4-ab7b-801a3cbd8aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:25.726359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167b3bb-ec04-49d4-ab7b-801a3cbd8aff admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:25.726739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167b3bb-ec04-49d4-ab7b-801a3cbd8aff admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:25.727484 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167b3bb-ec04-49d4-ab7b-801a3cbd8aff admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:25.728566 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167b3bb-ec04-49d4-ab7b-801a3cbd8aff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:25.731412 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167b3bb-ec04-49d4-ab7b-801a3cbd8aff admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:25.811702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1311/2654] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:25 2022] POST /identity/v3/projects => generated 377 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:25.816751 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dce87-b885-4fb7-b76f-5d7dc3ed2909 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:25.831082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dce87-b885-4fb7-b76f-5d7dc3ed2909 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:25.930794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2666a-2147-44af-9a14-09f57bc38cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:25.931615 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2666a-2147-44af-9a14-09f57bc38cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:25.931990 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2666a-2147-44af-9a14-09f57bc38cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:25.932384 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2666a-2147-44af-9a14-09f57bc38cf4 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:25.933760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2666a-2147-44af-9a14-09f57bc38cf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:25.937330 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2666a-2147-44af-9a14-09f57bc38cf4 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:26.044707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1344/2655] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:25 2022] POST /identity/v3/users => generated 289 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:26.046291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167b3bb-ec04-49d4-ab7b-801a3cbd8aff admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:26.059614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167b3bb-ec04-49d4-ab7b-801a3cbd8aff admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:26.064812 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30088cd7-60ab-4d3e-abff-cf1c3c3f6a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:26.065458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30088cd7-60ab-4d3e-abff-cf1c3c3f6a9d admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:26.065740 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30088cd7-60ab-4d3e-abff-cf1c3c3f6a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:26.066015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30088cd7-60ab-4d3e-abff-cf1c3c3f6a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:26.066708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30088cd7-60ab-4d3e-abff-cf1c3c3f6a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:26.068645 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30088cd7-60ab-4d3e-abff-cf1c3c3f6a9d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:26.082874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1312/2656] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:26.087410 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2666a-2147-44af-9a14-09f57bc38cf4 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:26.114030 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2666a-2147-44af-9a14-09f57bc38cf4 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:26.120705 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f3dc6-aa01-4d54-9806-fec2d07f3c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:26.121759 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f3dc6-aa01-4d54-9806-fec2d07f3c12 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:26.122371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f3dc6-aa01-4d54-9806-fec2d07f3c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:26.122677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f3dc6-aa01-4d54-9806-fec2d07f3c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:26.134803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f3dc6-aa01-4d54-9806-fec2d07f3c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:26.134803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f3dc6-aa01-4d54-9806-fec2d07f3c12 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:26.144339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1345/2657] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:26.146005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30088cd7-60ab-4d3e-abff-cf1c3c3f6a9d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:26.159784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30088cd7-60ab-4d3e-abff-cf1c3c3f6a9d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:26.167413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d991aba-493e-4fd8-8852-4886401b5357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:26.168372 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d991aba-493e-4fd8-8852-4886401b5357 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:26.169457 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d991aba-493e-4fd8-8852-4886401b5357 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:26.169843 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d991aba-493e-4fd8-8852-4886401b5357 admin admin] PATH_INFO: `/v3/projects/0527e71d814d4ba08ce0504bc06a073f/users/c758a87c114b4b90a4eacc5af1081e1c/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:26.199654 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d991aba-493e-4fd8-8852-4886401b5357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0527e71d814d4ba08ce0504bc06a073f, user_id=c758a87c114b4b90a4eacc5af1081e1c, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:26.202832 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d991aba-493e-4fd8-8852-4886401b5357 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:26.326992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1313/2658] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:26 2022] PUT /identity/v3/projects/0527e71d814d4ba08ce0504bc06a073f/users/c758a87c114b4b90a4eacc5af1081e1c/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:26.344678 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddf498-b0e1-4b0c-b1f6-a998f106ce4c None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:26.348671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddf498-b0e1-4b0c-b1f6-a998f106ce4c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:26.348671 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddf498-b0e1-4b0c-b1f6-a998f106ce4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:26.384015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d991aba-493e-4fd8-8852-4886401b5357 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:26.386473 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4d991aba-493e-4fd8-8852-4886401b5357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:26.407264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-44ddf498-b0e1-4b0c-b1f6-a998f106ce4c None None] MFA Rules not processed for user `c758a87c114b4b90a4eacc5af1081e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:26.409779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-44ddf498-b0e1-4b0c-b1f6-a998f106ce4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:26.505864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1346/2659] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:26.510760 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddf498-b0e1-4b0c-b1f6-a998f106ce4c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:26.521765 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d991aba-493e-4fd8-8852-4886401b5357 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:26.535002 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c774b-8fd2-4213-9b4a-9a5d85a7d9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:26.535653 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c774b-8fd2-4213-9b4a-9a5d85a7d9b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:26.535928 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c774b-8fd2-4213-9b4a-9a5d85a7d9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:26.536198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c774b-8fd2-4213-9b4a-9a5d85a7d9b2 admin admin] PATH_INFO: `/v3/users/a0bb160c9ea3458a9345e3c3a9186079` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:26.540405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddf498-b0e1-4b0c-b1f6-a998f106ce4c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:26.541143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c774b-8fd2-4213-9b4a-9a5d85a7d9b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0bb160c9ea3458a9345e3c3a9186079)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:26.546649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c774b-8fd2-4213-9b4a-9a5d85a7d9b2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:26.553465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1bf92-18f5-485e-889e-278b57c5f43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:26.554788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1bf92-18f5-485e-889e-278b57c5f43c admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:26.555269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1bf92-18f5-485e-889e-278b57c5f43c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:26.556387 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1bf92-18f5-485e-889e-278b57c5f43c admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:26.558594 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1bf92-18f5-485e-889e-278b57c5f43c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:26.563843 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1bf92-18f5-485e-889e-278b57c5f43c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:26.583377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:26.692417 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1347/2660] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:26 2022] POST /identity/v3/projects => generated 375 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:26.694028 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1bf92-18f5-485e-889e-278b57c5f43c admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:26.714403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:26.770939 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-606c774b-8fd2-4213-9b4a-9a5d85a7d9b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0bb160c9ea3458a9345e3c3a9186079'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:26.796457 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1bf92-18f5-485e-889e-278b57c5f43c admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:26.803379 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b2e0c-e7a5-4078-ab88-c687a5418a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:26.804231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b2e0c-e7a5-4078-ab88-c687a5418a1c admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:26.804543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b2e0c-e7a5-4078-ab88-c687a5418a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:26.804833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b2e0c-e7a5-4078-ab88-c687a5418a1c admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:26.805545 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b2e0c-e7a5-4078-ab88-c687a5418a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:26.808808 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b2e0c-e7a5-4078-ab88-c687a5418a1c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:26.889497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-606c774b-8fd2-4213-9b4a-9a5d85a7d9b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0bb160c9ea3458a9345e3c3a9186079'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:26.950854 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-606c774b-8fd2-4213-9b4a-9a5d85a7d9b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0bb160c9ea3458a9345e3c3a9186079'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:26.951986 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-606c774b-8fd2-4213-9b4a-9a5d85a7d9b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0bb160c9ea3458a9345e3c3a9186079'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:26.958282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1348/2661] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:26 2022] POST /identity/v3/users => generated 294 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:26.962779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b2e0c-e7a5-4078-ab88-c687a5418a1c admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:26.966866 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3c8b2e0c-e7a5-4078-ab88-c687a5418a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:26.976987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-606c774b-8fd2-4213-9b4a-9a5d85a7d9b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0bb160c9ea3458a9345e3c3a9186079'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:26.989406 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b2e0c-e7a5-4078-ab88-c687a5418a1c admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:26.995891 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e02f2-0ca7-4cf0-ae69-c51d933d94a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:26.996774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1314/2662] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:26 2022] DELETE /identity/v3/users/a0bb160c9ea3458a9345e3c3a9186079 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:27.000131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e02f2-0ca7-4cf0-ae69-c51d933d94a2 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:27.000131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e02f2-0ca7-4cf0-ae69-c51d933d94a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:27.000131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e02f2-0ca7-4cf0-ae69-c51d933d94a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:27.000131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e02f2-0ca7-4cf0-ae69-c51d933d94a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:27.002147 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0965e-c6c3-463e-b78f-35b603d6dbcf None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:27.002507 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0965e-c6c3-463e-b78f-35b603d6dbcf None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:27.002854 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e02f2-0ca7-4cf0-ae69-c51d933d94a2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:27.003068 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0965e-c6c3-463e-b78f-35b603d6dbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:27.025258 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1349/2663] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:27.026232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e02f2-0ca7-4cf0-ae69-c51d933d94a2 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:27.039909 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e02f2-0ca7-4cf0-ae69-c51d933d94a2 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:27.061122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-16a0965e-c6c3-463e-b78f-35b603d6dbcf None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:27.071039 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-16a0965e-c6c3-463e-b78f-35b603d6dbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:27.149606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f721047-12be-46db-af74-e99e8b0cb5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:27.150292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f721047-12be-46db-af74-e99e8b0cb5ea admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:27.150575 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f721047-12be-46db-af74-e99e8b0cb5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:27.150851 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f721047-12be-46db-af74-e99e8b0cb5ea admin admin] PATH_INFO: `/v3/projects/8341a94475e9440680b587e43023724f/users/e5a2b063e0f94f19aea009fc12859e66/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:27.181139 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1315/2664] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:27.186741 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f721047-12be-46db-af74-e99e8b0cb5ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8341a94475e9440680b587e43023724f, user_id=e5a2b063e0f94f19aea009fc12859e66, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:27.189765 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f721047-12be-46db-af74-e99e8b0cb5ea admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:27.194493 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0965e-c6c3-463e-b78f-35b603d6dbcf None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:27.329552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-16a0965e-c6c3-463e-b78f-35b603d6dbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:27.329552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0965e-c6c3-463e-b78f-35b603d6dbcf None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:27.329552 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c780b-1952-4443-bbdc-5730b8eda749 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:27.330016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c780b-1952-4443-bbdc-5730b8eda749 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:27.330016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c780b-1952-4443-bbdc-5730b8eda749 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:27.330016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c780b-1952-4443-bbdc-5730b8eda749 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:27.330016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c780b-1952-4443-bbdc-5730b8eda749 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:27.330016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c780b-1952-4443-bbdc-5730b8eda749 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:27.330016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1316/2665] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:08:27.457387 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1350/2666] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:27 2022] PUT /identity/v3/projects/8341a94475e9440680b587e43023724f/users/e5a2b063e0f94f19aea009fc12859e66/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:27.460459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c780b-1952-4443-bbdc-5730b8eda749 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:27.471954 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c780b-1952-4443-bbdc-5730b8eda749 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:27.543110 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:27.567590 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbb739-e9eb-4bd6-81b7-897aa6b4f6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:27.568412 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbb739-e9eb-4bd6-81b7-897aa6b4f6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:27.568682 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbb739-e9eb-4bd6-81b7-897aa6b4f6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:27.568997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbb739-e9eb-4bd6-81b7-897aa6b4f6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:27.570024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbb739-e9eb-4bd6-81b7-897aa6b4f6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:27.571336 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbb739-e9eb-4bd6-81b7-897aa6b4f6f8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:27.583652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1317/2667] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:27.587749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f721047-12be-46db-af74-e99e8b0cb5ea admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:27.608013 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f721047-12be-46db-af74-e99e8b0cb5ea admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:27.618049 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca5d4a-7150-4009-bd6f-b6e7f2ec9af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:27.619407 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca5d4a-7150-4009-bd6f-b6e7f2ec9af4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:27.619690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca5d4a-7150-4009-bd6f-b6e7f2ec9af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:27.619961 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca5d4a-7150-4009-bd6f-b6e7f2ec9af4 admin admin] PATH_INFO: `/v3/projects/8341a94475e9440680b587e43023724f/users/e5a2b063e0f94f19aea009fc12859e66/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:27.622164 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca5d4a-7150-4009-bd6f-b6e7f2ec9af4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8341a94475e9440680b587e43023724f, user_id=e5a2b063e0f94f19aea009fc12859e66, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:27.626134 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca5d4a-7150-4009-bd6f-b6e7f2ec9af4 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:27.664073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1351/2668] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:27 2022] PUT /identity/v3/projects/8341a94475e9440680b587e43023724f/users/e5a2b063e0f94f19aea009fc12859e66/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:27.665528 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17dcfa-f803-4a86-9203-016dbbc9ba07 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:27.666625 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17dcfa-f803-4a86-9203-016dbbc9ba07 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:27.667284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17dcfa-f803-4a86-9203-016dbbc9ba07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:27.735673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-cd17dcfa-f803-4a86-9203-016dbbc9ba07 None None] MFA Rules not processed for user `e5a2b063e0f94f19aea009fc12859e66`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:27.738733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-cd17dcfa-f803-4a86-9203-016dbbc9ba07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:27.928612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1318/2669] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:27.941939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca5d4a-7150-4009-bd6f-b6e7f2ec9af4 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:27.943779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-aaca5d4a-7150-4009-bd6f-b6e7f2ec9af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:28.011390 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17dcfa-f803-4a86-9203-016dbbc9ba07 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:28.012872 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-cd17dcfa-f803-4a86-9203-016dbbc9ba07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:28.023140 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca5d4a-7150-4009-bd6f-b6e7f2ec9af4 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:28.028596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bbd15-0c1d-4fd2-b02a-b20153bf6809 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:28.029285 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bbd15-0c1d-4fd2-b02a-b20153bf6809 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:28.029932 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bbd15-0c1d-4fd2-b02a-b20153bf6809 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:28.030212 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bbd15-0c1d-4fd2-b02a-b20153bf6809 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:28.048438 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3c3bbd15-0c1d-4fd2-b02a-b20153bf6809 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:28.068721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:28.145447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bbd15-0c1d-4fd2-b02a-b20153bf6809 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:28.147780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bbd15-0c1d-4fd2-b02a-b20153bf6809 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:28.168221 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17dcfa-f803-4a86-9203-016dbbc9ba07 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:28.177908 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1352/2670] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:28.192455 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfa236-26bb-4c6e-93f4-2e16944f6925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:28.193086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfa236-26bb-4c6e-93f4-2e16944f6925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:28.193420 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfa236-26bb-4c6e-93f4-2e16944f6925 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:28.193707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dfa236-26bb-4c6e-93f4-2e16944f6925 admin admin] PATH_INFO: `/v3/projects/48c539bd128e4637ba60f521f7dbba9b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:28.197136 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfa236-26bb-4c6e-93f4-2e16944f6925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48c539bd128e4637ba60f521f7dbba9b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:28.204361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dfa236-26bb-4c6e-93f4-2e16944f6925 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:28.337418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e5dfa236-26bb-4c6e-93f4-2e16944f6925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48c539bd128e4637ba60f521f7dbba9b', 'request_id': 'req-e5dfa236-26bb-4c6e-93f4-2e16944f6925'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:28.352964 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e5dfa236-26bb-4c6e-93f4-2e16944f6925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48c539bd128e4637ba60f521f7dbba9b', 'request_id': 'req-e5dfa236-26bb-4c6e-93f4-2e16944f6925'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:28.363851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e5dfa236-26bb-4c6e-93f4-2e16944f6925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48c539bd128e4637ba60f521f7dbba9b', 'request_id': 'req-e5dfa236-26bb-4c6e-93f4-2e16944f6925'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:28.368201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:28.470182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e5dfa236-26bb-4c6e-93f4-2e16944f6925 admin admin] The token cache is being invalidate because project 48c539bd128e4637ba60f521f7dbba9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:28.470698 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e5dfa236-26bb-4c6e-93f4-2e16944f6925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:28.475713 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e5dfa236-26bb-4c6e-93f4-2e16944f6925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:28.482123 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1319/2671] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:28 2022] DELETE /identity/v3/projects/48c539bd128e4637ba60f521f7dbba9b => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:28.483203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bbd15-0c1d-4fd2-b02a-b20153bf6809 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:28.495606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bbd15-0c1d-4fd2-b02a-b20153bf6809 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:28.584068 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05a94a-4454-4459-a13f-d8cb50d8ebb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:28.584706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05a94a-4454-4459-a13f-d8cb50d8ebb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:28.584984 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05a94a-4454-4459-a13f-d8cb50d8ebb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:28.587439 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05a94a-4454-4459-a13f-d8cb50d8ebb8 admin admin] PATH_INFO: `/v3/users/04af9d0b99f54f7a9951617ccca812fa` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:28.592615 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05a94a-4454-4459-a13f-d8cb50d8ebb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04af9d0b99f54f7a9951617ccca812fa)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:28.596843 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b05a94a-4454-4459-a13f-d8cb50d8ebb8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:28.881480 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8b05a94a-4454-4459-a13f-d8cb50d8ebb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04af9d0b99f54f7a9951617ccca812fa'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:28.983066 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8b05a94a-4454-4459-a13f-d8cb50d8ebb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04af9d0b99f54f7a9951617ccca812fa'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:28.984013 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8b05a94a-4454-4459-a13f-d8cb50d8ebb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04af9d0b99f54f7a9951617ccca812fa'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:28.996862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8b05a94a-4454-4459-a13f-d8cb50d8ebb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04af9d0b99f54f7a9951617ccca812fa'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:29.054409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8b05a94a-4454-4459-a13f-d8cb50d8ebb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04af9d0b99f54f7a9951617ccca812fa'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:29.092173 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1353/2672] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:28 2022] DELETE /identity/v3/users/04af9d0b99f54f7a9951617ccca812fa => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:29.829396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfa236-26bb-4c6e-93f4-2e16944f6925 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:29.831192 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e5dfa236-26bb-4c6e-93f4-2e16944f6925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:29.852496 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dfa236-26bb-4c6e-93f4-2e16944f6925 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:29.957659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:29.958384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:29.958730 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:29.959086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6 admin admin] PATH_INFO: `/v3/projects/d0fa837cefcf44f1a5fc12dd19756991` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:29.960397 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0fa837cefcf44f1a5fc12dd19756991)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:29.963346 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:30.119374 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0fa837cefcf44f1a5fc12dd19756991', 'request_id': 'req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:30.130095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0fa837cefcf44f1a5fc12dd19756991', 'request_id': 'req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:30.138853 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0fa837cefcf44f1a5fc12dd19756991', 'request_id': 'req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:30.229851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6 admin admin] The token cache is being invalidate because project d0fa837cefcf44f1a5fc12dd19756991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:30.230863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:30.233944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:30.240095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1320/2673] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:29 2022] DELETE /identity/v3/projects/d0fa837cefcf44f1a5fc12dd19756991 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:30.244203 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05a94a-4454-4459-a13f-d8cb50d8ebb8 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:30.246046 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8b05a94a-4454-4459-a13f-d8cb50d8ebb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:30.267835 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b05a94a-4454-4459-a13f-d8cb50d8ebb8 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:30.333612 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:30.382022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaae88d-6018-4cad-8fd5-4eb46a49f92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:30.383079 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaae88d-6018-4cad-8fd5-4eb46a49f92c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:30.383788 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaae88d-6018-4cad-8fd5-4eb46a49f92c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:30.384494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaae88d-6018-4cad-8fd5-4eb46a49f92c admin admin] PATH_INFO: `/v3/users/f2324a64a858460ca6fa6c0e6f6f0461` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:30.386431 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaae88d-6018-4cad-8fd5-4eb46a49f92c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2324a64a858460ca6fa6c0e6f6f0461)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:30.389200 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaae88d-6018-4cad-8fd5-4eb46a49f92c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:30.495774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:30.497991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:30.596728 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4eaae88d-6018-4cad-8fd5-4eb46a49f92c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2324a64a858460ca6fa6c0e6f6f0461'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:30.607451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e729d39-8b91-4693-8a76-0e3ea7f0f7a6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:30.611754 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd4a66-be1e-4d01-b7e3-08140b4b6c20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:30.612583 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd4a66-be1e-4d01-b7e3-08140b4b6c20 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:30.612960 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd4a66-be1e-4d01-b7e3-08140b4b6c20 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:30.613377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd4a66-be1e-4d01-b7e3-08140b4b6c20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:30.615382 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f9fd4a66-be1e-4d01-b7e3-08140b4b6c20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:30.663607 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:30.686300 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4eaae88d-6018-4cad-8fd5-4eb46a49f92c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2324a64a858460ca6fa6c0e6f6f0461'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:30.687747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4eaae88d-6018-4cad-8fd5-4eb46a49f92c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2324a64a858460ca6fa6c0e6f6f0461'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:30.692262 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd4a66-be1e-4d01-b7e3-08140b4b6c20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:30.694038 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd4a66-be1e-4d01-b7e3-08140b4b6c20 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:30.711921 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4eaae88d-6018-4cad-8fd5-4eb46a49f92c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2324a64a858460ca6fa6c0e6f6f0461'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:30.712954 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1321/2674] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:08:30.714059 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd4a66-be1e-4d01-b7e3-08140b4b6c20 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:30.716710 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f9fd4a66-be1e-4d01-b7e3-08140b4b6c20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:30.735831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd4a66-be1e-4d01-b7e3-08140b4b6c20 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:30.745022 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3070c7a-b000-4064-8a68-c09e9f45a6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:30.745668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3070c7a-b000-4064-8a68-c09e9f45a6dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:30.745950 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3070c7a-b000-4064-8a68-c09e9f45a6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:30.747594 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3070c7a-b000-4064-8a68-c09e9f45a6dc admin admin] PATH_INFO: `/v3/users/48bf6dab289545bb80182c8904c05d6c` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:30.753830 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-4eaae88d-6018-4cad-8fd5-4eb46a49f92c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2324a64a858460ca6fa6c0e6f6f0461'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:30.755166 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3070c7a-b000-4064-8a68-c09e9f45a6dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=48bf6dab289545bb80182c8904c05d6c)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:30.757129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3070c7a-b000-4064-8a68-c09e9f45a6dc admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:30.764802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1354/2675] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:30 2022] DELETE /identity/v3/users/f2324a64a858460ca6fa6c0e6f6f0461 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:30.966635 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f3070c7a-b000-4064-8a68-c09e9f45a6dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48bf6dab289545bb80182c8904c05d6c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:31.079614 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f3070c7a-b000-4064-8a68-c09e9f45a6dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48bf6dab289545bb80182c8904c05d6c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:31.109663 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f3070c7a-b000-4064-8a68-c09e9f45a6dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48bf6dab289545bb80182c8904c05d6c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:31.112236 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f3070c7a-b000-4064-8a68-c09e9f45a6dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48bf6dab289545bb80182c8904c05d6c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:31.135406 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f3070c7a-b000-4064-8a68-c09e9f45a6dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48bf6dab289545bb80182c8904c05d6c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:31.146718 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1322/2676] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:30 2022] DELETE /identity/v3/users/48bf6dab289545bb80182c8904c05d6c => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:31.595062 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaae88d-6018-4cad-8fd5-4eb46a49f92c admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:31.596490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4eaae88d-6018-4cad-8fd5-4eb46a49f92c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:31.652519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3070c7a-b000-4064-8a68-c09e9f45a6dc admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:31.655123 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f3070c7a-b000-4064-8a68-c09e9f45a6dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:31.681664 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaae88d-6018-4cad-8fd5-4eb46a49f92c admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:31.770162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21eafb-166e-430a-8dc6-26101687e849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:31.770821 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21eafb-166e-430a-8dc6-26101687e849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:31.771251 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21eafb-166e-430a-8dc6-26101687e849 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:31.771528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21eafb-166e-430a-8dc6-26101687e849 admin admin] PATH_INFO: `/v3/projects/b845777a282e46ff8c5a95d7aa967ec5` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:31.774206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e21eafb-166e-430a-8dc6-26101687e849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b845777a282e46ff8c5a95d7aa967ec5)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:31.776210 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e21eafb-166e-430a-8dc6-26101687e849 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:31.782822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:31.790142 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3070c7a-b000-4064-8a68-c09e9f45a6dc admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:31.795747 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe312d3-e830-4a68-bf53-fd5d29b0c9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:31.796897 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe312d3-e830-4a68-bf53-fd5d29b0c9b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:31.797738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe312d3-e830-4a68-bf53-fd5d29b0c9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:31.798253 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe312d3-e830-4a68-bf53-fd5d29b0c9b7 admin admin] PATH_INFO: `/v3/projects/a9448ca7c8234f74b24c8f6710a0ef5c` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:31.799821 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe312d3-e830-4a68-bf53-fd5d29b0c9b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9448ca7c8234f74b24c8f6710a0ef5c)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:31.801967 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe312d3-e830-4a68-bf53-fd5d29b0c9b7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:31.908010 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7e21eafb-166e-430a-8dc6-26101687e849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b845777a282e46ff8c5a95d7aa967ec5', 'request_id': 'req-7e21eafb-166e-430a-8dc6-26101687e849'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:31.942772 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3fe312d3-e830-4a68-bf53-fd5d29b0c9b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9448ca7c8234f74b24c8f6710a0ef5c', 'request_id': 'req-3fe312d3-e830-4a68-bf53-fd5d29b0c9b7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:31.945517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:31.950162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3fe312d3-e830-4a68-bf53-fd5d29b0c9b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9448ca7c8234f74b24c8f6710a0ef5c', 'request_id': 'req-3fe312d3-e830-4a68-bf53-fd5d29b0c9b7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:31.959638 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3fe312d3-e830-4a68-bf53-fd5d29b0c9b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9448ca7c8234f74b24c8f6710a0ef5c', 'request_id': 'req-3fe312d3-e830-4a68-bf53-fd5d29b0c9b7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:31.973980 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7e21eafb-166e-430a-8dc6-26101687e849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b845777a282e46ff8c5a95d7aa967ec5', 'request_id': 'req-7e21eafb-166e-430a-8dc6-26101687e849'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:31.985710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7e21eafb-166e-430a-8dc6-26101687e849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b845777a282e46ff8c5a95d7aa967ec5', 'request_id': 'req-7e21eafb-166e-430a-8dc6-26101687e849'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:31.999079 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7e21eafb-166e-430a-8dc6-26101687e849 admin admin] The token cache is being invalidate because project b845777a282e46ff8c5a95d7aa967ec5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:31.999665 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7e21eafb-166e-430a-8dc6-26101687e849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:32.000606 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7e21eafb-166e-430a-8dc6-26101687e849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:32.003346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1355/2677] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:31 2022] DELETE /identity/v3/projects/b845777a282e46ff8c5a95d7aa967ec5 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:32.011331 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c8f01-4fc9-4cf7-a026-43f64a8d76d4 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:32.011834 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c8f01-4fc9-4cf7-a026-43f64a8d76d4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:32.012354 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c8f01-4fc9-4cf7-a026-43f64a8d76d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:32.040560 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-202c8f01-4fc9-4cf7-a026-43f64a8d76d4 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:32.041931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-202c8f01-4fc9-4cf7-a026-43f64a8d76d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:32.099141 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3fe312d3-e830-4a68-bf53-fd5d29b0c9b7 admin admin] The token cache is being invalidate because project a9448ca7c8234f74b24c8f6710a0ef5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:32.099690 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3fe312d3-e830-4a68-bf53-fd5d29b0c9b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:32.100352 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3fe312d3-e830-4a68-bf53-fd5d29b0c9b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:32.102266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1323/2678] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:31 2022] DELETE /identity/v3/projects/a9448ca7c8234f74b24c8f6710a0ef5c => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:32.108886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e038b-f826-4468-b369-3d4e90f80b65 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:32.109299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e038b-f826-4468-b369-3d4e90f80b65 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:32.109658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e038b-f826-4468-b369-3d4e90f80b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:32.141062 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1356/2679] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:32.154479 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f50e038b-f826-4468-b369-3d4e90f80b65 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:32.155359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76716d17-62b6-4bf4-b802-47d0c566d3e6 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:32.155649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76716d17-62b6-4bf4-b802-47d0c566d3e6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:32.155924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76716d17-62b6-4bf4-b802-47d0c566d3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:32.157825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f50e038b-f826-4468-b369-3d4e90f80b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:32.169441 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1324/2680] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:32.181028 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-76716d17-62b6-4bf4-b802-47d0c566d3e6 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:32.182706 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-76716d17-62b6-4bf4-b802-47d0c566d3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:32.187239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967497a2-5754-45e6-8dfd-383190ce4b25 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:32.187752 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967497a2-5754-45e6-8dfd-383190ce4b25 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:32.188229 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967497a2-5754-45e6-8dfd-383190ce4b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:32.193747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1357/2681] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:32.198424 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76716d17-62b6-4bf4-b802-47d0c566d3e6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:32.208523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76716d17-62b6-4bf4-b802-47d0c566d3e6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:32.215345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f397bfb-8a01-477d-a17b-3db8e66f3e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:32.215971 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f397bfb-8a01-477d-a17b-3db8e66f3e5a admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:32.216254 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f397bfb-8a01-477d-a17b-3db8e66f3e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:32.216531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f397bfb-8a01-477d-a17b-3db8e66f3e5a admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:32.217363 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f397bfb-8a01-477d-a17b-3db8e66f3e5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:08:32.217670 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f397bfb-8a01-477d-a17b-3db8e66f3e5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:32.220129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f397bfb-8a01-477d-a17b-3db8e66f3e5a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:32.232354 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-967497a2-5754-45e6-8dfd-383190ce4b25 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:32.239272 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1358/2682] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:08:32.239437 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-967497a2-5754-45e6-8dfd-383190ce4b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:32.239623 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f397bfb-8a01-477d-a17b-3db8e66f3e5a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:32.253863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1325/2683] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:32.254106 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f397bfb-8a01-477d-a17b-3db8e66f3e5a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:32.255154 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967497a2-5754-45e6-8dfd-383190ce4b25 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:32.258957 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c151f413-d96a-4bfd-bc36-ec13fcfd2b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:32.259946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151f413-d96a-4bfd-bc36-ec13fcfd2b21 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:32.260508 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151f413-d96a-4bfd-bc36-ec13fcfd2b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:32.261937 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c151f413-d96a-4bfd-bc36-ec13fcfd2b21 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:32.263313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c151f413-d96a-4bfd-bc36-ec13fcfd2b21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:32.265598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c151f413-d96a-4bfd-bc36-ec13fcfd2b21 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:32.270128 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967497a2-5754-45e6-8dfd-383190ce4b25 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:32.275463 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8627bf-5632-44d5-8948-cae1e01afec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:32.276168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8627bf-5632-44d5-8948-cae1e01afec8 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:32.276517 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8627bf-5632-44d5-8948-cae1e01afec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:32.276912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8627bf-5632-44d5-8948-cae1e01afec8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:32.277664 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8627bf-5632-44d5-8948-cae1e01afec8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:08:32.278033 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8627bf-5632-44d5-8948-cae1e01afec8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:32.279733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8627bf-5632-44d5-8948-cae1e01afec8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:32.295667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1326/2684] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:08:32.300087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8627bf-5632-44d5-8948-cae1e01afec8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:32.313361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8627bf-5632-44d5-8948-cae1e01afec8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:32.318362 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdec479-3fa6-4cac-9a9b-b3c5c3d7e015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:32.319005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdec479-3fa6-4cac-9a9b-b3c5c3d7e015 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:32.319288 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdec479-3fa6-4cac-9a9b-b3c5c3d7e015 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:32.319564 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdec479-3fa6-4cac-9a9b-b3c5c3d7e015 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:32.320197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdec479-3fa6-4cac-9a9b-b3c5c3d7e015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:32.322093 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdec479-3fa6-4cac-9a9b-b3c5c3d7e015 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:32.403082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1359/2685] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:32 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:32.404052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c151f413-d96a-4bfd-bc36-ec13fcfd2b21 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:32.463917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1327/2686] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:32 2022] POST /identity/v3/projects => generated 383 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:32.468308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdec479-3fa6-4cac-9a9b-b3c5c3d7e015 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:32.490060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c151f413-d96a-4bfd-bc36-ec13fcfd2b21 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:32.560377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdec479-3fa6-4cac-9a9b-b3c5c3d7e015 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:32.570787 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353faef-e364-406e-be02-61d87cd8c928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:32.571948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353faef-e364-406e-be02-61d87cd8c928 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:32.572329 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353faef-e364-406e-be02-61d87cd8c928 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:32.572679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353faef-e364-406e-be02-61d87cd8c928 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:32.573380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353faef-e364-406e-be02-61d87cd8c928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:32.576646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353faef-e364-406e-be02-61d87cd8c928 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:32.595087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3e7fb-860d-41ae-a782-2e9a881d5a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:32.600054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3e7fb-860d-41ae-a782-2e9a881d5a54 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:32.600054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3e7fb-860d-41ae-a782-2e9a881d5a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:32.600054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3e7fb-860d-41ae-a782-2e9a881d5a54 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:32.600054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3e7fb-860d-41ae-a782-2e9a881d5a54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:32.600054 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3e7fb-860d-41ae-a782-2e9a881d5a54 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:32.682987 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1360/2687] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:32 2022] POST /identity/v3/users => generated 293 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:32.689279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3e7fb-860d-41ae-a782-2e9a881d5a54 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:32.689279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:32.690705 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1328/2688] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:32 2022] POST /identity/v3/users => generated 292 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:32.691929 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353faef-e364-406e-be02-61d87cd8c928 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:32.707758 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3e7fb-860d-41ae-a782-2e9a881d5a54 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:32.708223 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353faef-e364-406e-be02-61d87cd8c928 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:32.716582 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe411f-6a40-4c82-bfaa-a2b3fa5d6534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:32.717259 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe411f-6a40-4c82-bfaa-a2b3fa5d6534 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:32.717450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe411f-6a40-4c82-bfaa-a2b3fa5d6534 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:32.717725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe411f-6a40-4c82-bfaa-a2b3fa5d6534 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:32.718553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe411f-6a40-4c82-bfaa-a2b3fa5d6534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:32.719300 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95231f-98ad-4a87-93c7-5d751b7c86f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:32.720103 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95231f-98ad-4a87-93c7-5d751b7c86f2 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:32.720342 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe411f-6a40-4c82-bfaa-a2b3fa5d6534 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:32.720571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95231f-98ad-4a87-93c7-5d751b7c86f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:32.720953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95231f-98ad-4a87-93c7-5d751b7c86f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:32.721779 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95231f-98ad-4a87-93c7-5d751b7c86f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:32.723978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95231f-98ad-4a87-93c7-5d751b7c86f2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:32.734177 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1329/2689] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:32.752303 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe411f-6a40-4c82-bfaa-a2b3fa5d6534 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:32.766974 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe411f-6a40-4c82-bfaa-a2b3fa5d6534 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:32.768178 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1361/2690] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:32.769986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95231f-98ad-4a87-93c7-5d751b7c86f2 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:32.781397 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeda8a6-51ae-4635-bffe-dad4ed5a6211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:32.782763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeda8a6-51ae-4635-bffe-dad4ed5a6211 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:32.783162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeda8a6-51ae-4635-bffe-dad4ed5a6211 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:32.783431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeda8a6-51ae-4635-bffe-dad4ed5a6211 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:32.784239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeda8a6-51ae-4635-bffe-dad4ed5a6211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:32.786172 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeda8a6-51ae-4635-bffe-dad4ed5a6211 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:32.786616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95231f-98ad-4a87-93c7-5d751b7c86f2 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:32.796343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355a57b-26d0-4e48-97e6-6c0b7efa3655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:32.797316 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355a57b-26d0-4e48-97e6-6c0b7efa3655 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:32.797759 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355a57b-26d0-4e48-97e6-6c0b7efa3655 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:32.798192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355a57b-26d0-4e48-97e6-6c0b7efa3655 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:32.799029 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355a57b-26d0-4e48-97e6-6c0b7efa3655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:32.800794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355a57b-26d0-4e48-97e6-6c0b7efa3655 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:32.804815 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1330/2691] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:32.806161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeda8a6-51ae-4635-bffe-dad4ed5a6211 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:32.809219 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-cfeda8a6-51ae-4635-bffe-dad4ed5a6211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:32.816248 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1362/2692] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:32.816428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355a57b-26d0-4e48-97e6-6c0b7efa3655 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:32.833099 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355a57b-26d0-4e48-97e6-6c0b7efa3655 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:32.841290 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe2511-b151-4b58-a6e2-6ff81e284884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:32.842327 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe2511-b151-4b58-a6e2-6ff81e284884 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:32.842815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe2511-b151-4b58-a6e2-6ff81e284884 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:32.843339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe2511-b151-4b58-a6e2-6ff81e284884 admin admin] PATH_INFO: `/v3/projects/ea4a774193494f7eab5047a98a285a7c/users/206c96a9fff349bdbe63ff045cfe74cf/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:32.866053 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe2511-b151-4b58-a6e2-6ff81e284884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea4a774193494f7eab5047a98a285a7c, user_id=206c96a9fff349bdbe63ff045cfe74cf, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:32.870153 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe2511-b151-4b58-a6e2-6ff81e284884 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:32.878462 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeda8a6-51ae-4635-bffe-dad4ed5a6211 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:32.883275 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87aefc8-98e4-4f1d-a064-06ab6fddf63a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:32.883892 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87aefc8-98e4-4f1d-a064-06ab6fddf63a service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:32.884161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87aefc8-98e4-4f1d-a064-06ab6fddf63a service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:32.884430 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87aefc8-98e4-4f1d-a064-06ab6fddf63a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:32.887043 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b87aefc8-98e4-4f1d-a064-06ab6fddf63a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:32.904637 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:32.946342 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1363/2693] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:32 2022] PUT /identity/v3/projects/ea4a774193494f7eab5047a98a285a7c/users/206c96a9fff349bdbe63ff045cfe74cf/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:32.947248 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe2511-b151-4b58-a6e2-6ff81e284884 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:32.970314 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87aefc8-98e4-4f1d-a064-06ab6fddf63a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:32.972373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87aefc8-98e4-4f1d-a064-06ab6fddf63a service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:32.986073 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1331/2694] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:32.998811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef3dd3-7295-4e30-ba1f-8a8ad8c245b3 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:33.002432 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef3dd3-7295-4e30-ba1f-8a8ad8c245b3 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:33.002923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef3dd3-7295-4e30-ba1f-8a8ad8c245b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:33.024064 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe2511-b151-4b58-a6e2-6ff81e284884 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:33.032613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c95cd-d6ce-4692-814e-70735496b2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:33.033805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c95cd-d6ce-4692-814e-70735496b2e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:33.034586 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c95cd-d6ce-4692-814e-70735496b2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:33.034887 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c95cd-d6ce-4692-814e-70735496b2e6 admin admin] PATH_INFO: `/v3/projects/ad68c575a69c47c099877ba0d338662d/users/caf4997eadb14794890f47ded5aeea7b/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:33.054793 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-26ef3dd3-7295-4e30-ba1f-8a8ad8c245b3 None None] MFA Rules not processed for user `206c96a9fff349bdbe63ff045cfe74cf`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:33.055934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c95cd-d6ce-4692-814e-70735496b2e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad68c575a69c47c099877ba0d338662d, user_id=caf4997eadb14794890f47ded5aeea7b, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:33.056971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-26ef3dd3-7295-4e30-ba1f-8a8ad8c245b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:33.059018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c95cd-d6ce-4692-814e-70735496b2e6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:33.139209 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1364/2695] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:32 2022] PUT /identity/v3/projects/ad68c575a69c47c099877ba0d338662d/users/caf4997eadb14794890f47ded5aeea7b/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:33.145073 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1332/2696] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:33.185384 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c95cd-d6ce-4692-814e-70735496b2e6 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:33.197014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-402c95cd-d6ce-4692-814e-70735496b2e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:33.211698 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d7cdb-bb81-484e-8255-4bfbf7dbaef4 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:33.212294 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d7cdb-bb81-484e-8255-4bfbf7dbaef4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:33.212839 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d7cdb-bb81-484e-8255-4bfbf7dbaef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:33.261837 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:33.261957 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a85d7cdb-bb81-484e-8255-4bfbf7dbaef4 None None] MFA Rules not processed for user `caf4997eadb14794890f47ded5aeea7b`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:33.261957 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a85d7cdb-bb81-484e-8255-4bfbf7dbaef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:33.338941 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1333/2697] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:33.348027 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d7cdb-bb81-484e-8255-4bfbf7dbaef4 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:33.356199 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c95cd-d6ce-4692-814e-70735496b2e6 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:33.364469 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a00f3-8777-4bcb-83ef-5227979c2043 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:33.365090 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a00f3-8777-4bcb-83ef-5227979c2043 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:33.365372 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a00f3-8777-4bcb-83ef-5227979c2043 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:33.365648 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a00f3-8777-4bcb-83ef-5227979c2043 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:33.386534 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:33.414592 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d7cdb-bb81-484e-8255-4bfbf7dbaef4 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:33.419726 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef37d2-eff6-4673-898c-ce607a9b4758 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:33.420408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef37d2-eff6-4673-898c-ce607a9b4758 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:33.420586 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef37d2-eff6-4673-898c-ce607a9b4758 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:33.421086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ef37d2-eff6-4673-898c-ce607a9b4758 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:33.432249 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef37d2-eff6-4673-898c-ce607a9b4758 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:33.434844 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ef37d2-eff6-4673-898c-ce607a9b4758 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:33.445135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a00f3-8777-4bcb-83ef-5227979c2043 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:33.446400 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1334/2698] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:08:33.457627 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a00f3-8777-4bcb-83ef-5227979c2043 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:33.472591 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1365/2699] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:36.810102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef37d2-eff6-4673-898c-ce607a9b4758 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:36.911860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ef37d2-eff6-4673-898c-ce607a9b4758 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:36.920127 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762dc27-59eb-460b-89fc-a7b05ea33ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:36.920754 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762dc27-59eb-460b-89fc-a7b05ea33ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:36.921032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762dc27-59eb-460b-89fc-a7b05ea33ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:36.921360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762dc27-59eb-460b-89fc-a7b05ea33ba6 admin admin] PATH_INFO: `/v3/users/caf4997eadb14794890f47ded5aeea7b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:36.923317 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762dc27-59eb-460b-89fc-a7b05ea33ba6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caf4997eadb14794890f47ded5aeea7b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:36.925653 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762dc27-59eb-460b-89fc-a7b05ea33ba6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:37.235519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6762dc27-59eb-460b-89fc-a7b05ea33ba6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caf4997eadb14794890f47ded5aeea7b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:37.357264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6762dc27-59eb-460b-89fc-a7b05ea33ba6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caf4997eadb14794890f47ded5aeea7b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:37.382237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6762dc27-59eb-460b-89fc-a7b05ea33ba6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caf4997eadb14794890f47ded5aeea7b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:37.383644 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6762dc27-59eb-460b-89fc-a7b05ea33ba6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caf4997eadb14794890f47ded5aeea7b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:37.394139 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6762dc27-59eb-460b-89fc-a7b05ea33ba6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caf4997eadb14794890f47ded5aeea7b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:37.406551 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1335/2700] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:36 2022] DELETE /identity/v3/users/caf4997eadb14794890f47ded5aeea7b => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:37.409411 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4adf7c-6dba-4503-a0aa-7ed166216245 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:37.410055 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4adf7c-6dba-4503-a0aa-7ed166216245 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:37.410055 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4adf7c-6dba-4503-a0aa-7ed166216245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:37.436152 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3a4adf7c-6dba-4503-a0aa-7ed166216245 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:37.438062 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3a4adf7c-6dba-4503-a0aa-7ed166216245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:37.527257 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1366/2701] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:37.535806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762dc27-59eb-460b-89fc-a7b05ea33ba6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:37.537305 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6762dc27-59eb-460b-89fc-a7b05ea33ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:37.620018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762dc27-59eb-460b-89fc-a7b05ea33ba6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:37.624451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab82cf9-3a5d-4fe7-b02f-4e4b9d67e2a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:37.625067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab82cf9-3a5d-4fe7-b02f-4e4b9d67e2a0 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:37.625353 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab82cf9-3a5d-4fe7-b02f-4e4b9d67e2a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:37.625612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab82cf9-3a5d-4fe7-b02f-4e4b9d67e2a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:37.633299 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:37.639657 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab82cf9-3a5d-4fe7-b02f-4e4b9d67e2a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:37.641247 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab82cf9-3a5d-4fe7-b02f-4e4b9d67e2a0 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:37.653162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1336/2702] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:38.094549 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4adf7c-6dba-4503-a0aa-7ed166216245 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:38.096018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3a4adf7c-6dba-4503-a0aa-7ed166216245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:38.108118 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4adf7c-6dba-4503-a0aa-7ed166216245 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:38.113090 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c025586-cb0b-47f9-9f74-8697d0eec0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:38.113714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c025586-cb0b-47f9-9f74-8697d0eec0c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:38.113994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c025586-cb0b-47f9-9f74-8697d0eec0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:38.114264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c025586-cb0b-47f9-9f74-8697d0eec0c1 admin admin] PATH_INFO: `/v3/projects/ad68c575a69c47c099877ba0d338662d` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:38.115495 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c025586-cb0b-47f9-9f74-8697d0eec0c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad68c575a69c47c099877ba0d338662d)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:38.117441 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c025586-cb0b-47f9-9f74-8697d0eec0c1 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:38.212204 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:38.219101 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6c025586-cb0b-47f9-9f74-8697d0eec0c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad68c575a69c47c099877ba0d338662d', 'request_id': 'req-6c025586-cb0b-47f9-9f74-8697d0eec0c1'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:38.253910 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab82cf9-3a5d-4fe7-b02f-4e4b9d67e2a0 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:38.255377 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-aab82cf9-3a5d-4fe7-b02f-4e4b9d67e2a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:38.268351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab82cf9-3a5d-4fe7-b02f-4e4b9d67e2a0 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:38.296776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6c025586-cb0b-47f9-9f74-8697d0eec0c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad68c575a69c47c099877ba0d338662d', 'request_id': 'req-6c025586-cb0b-47f9-9f74-8697d0eec0c1'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:38.303837 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6c025586-cb0b-47f9-9f74-8697d0eec0c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad68c575a69c47c099877ba0d338662d', 'request_id': 'req-6c025586-cb0b-47f9-9f74-8697d0eec0c1'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:38.315712 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6c025586-cb0b-47f9-9f74-8697d0eec0c1 admin admin] The token cache is being invalidate because project ad68c575a69c47c099877ba0d338662d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:38.316106 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6c025586-cb0b-47f9-9f74-8697d0eec0c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:38.316819 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6c025586-cb0b-47f9-9f74-8697d0eec0c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:38.318637 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1367/2703] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:38 2022] DELETE /identity/v3/projects/ad68c575a69c47c099877ba0d338662d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:38.323149 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06776055-b4a9-480f-8d60-8d6cb068b6ee None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:38.323610 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06776055-b4a9-480f-8d60-8d6cb068b6ee None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:38.323805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06776055-b4a9-480f-8d60-8d6cb068b6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:38.328324 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7264646c-1b18-4ca1-a9a4-2d2e9bb5cb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:38.328932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264646c-1b18-4ca1-a9a4-2d2e9bb5cb93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:38.335643 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264646c-1b18-4ca1-a9a4-2d2e9bb5cb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:38.335643 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7264646c-1b18-4ca1-a9a4-2d2e9bb5cb93 admin admin] PATH_INFO: `/v3/users/d8eebf5044a64ff6b9b3d946ac2ab129` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:38.335643 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264646c-1b18-4ca1-a9a4-2d2e9bb5cb93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8eebf5044a64ff6b9b3d946ac2ab129)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:38.335643 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7264646c-1b18-4ca1-a9a4-2d2e9bb5cb93 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:38.344967 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-06776055-b4a9-480f-8d60-8d6cb068b6ee None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:38.346185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-06776055-b4a9-480f-8d60-8d6cb068b6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:38.355885 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1368/2704] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:38.363368 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdb96e-416f-4fe0-b928-9ad9db5a1344 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:38.364218 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdb96e-416f-4fe0-b928-9ad9db5a1344 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:38.364218 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdb96e-416f-4fe0-b928-9ad9db5a1344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:38.384129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1bcdb96e-416f-4fe0-b928-9ad9db5a1344 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:38.385248 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1bcdb96e-416f-4fe0-b928-9ad9db5a1344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:38.393990 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1369/2705] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:38.398221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdb96e-416f-4fe0-b928-9ad9db5a1344 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:38.408859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdb96e-416f-4fe0-b928-9ad9db5a1344 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:38.413418 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5279fe-cf83-4f31-9cae-8e6e564e111b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:38.414025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5279fe-cf83-4f31-9cae-8e6e564e111b admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:38.414256 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5279fe-cf83-4f31-9cae-8e6e564e111b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:38.414524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5279fe-cf83-4f31-9cae-8e6e564e111b admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:38.415183 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5279fe-cf83-4f31-9cae-8e6e564e111b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:08:38.415483 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5279fe-cf83-4f31-9cae-8e6e564e111b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:38.417064 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5279fe-cf83-4f31-9cae-8e6e564e111b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:38.428748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1370/2706] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:08:38.433007 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5279fe-cf83-4f31-9cae-8e6e564e111b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:38.442494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5279fe-cf83-4f31-9cae-8e6e564e111b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:38.447077 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3876c9-daa4-4718-a43d-9499f54c6f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:38.447785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3876c9-daa4-4718-a43d-9499f54c6f98 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:38.448058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3876c9-daa4-4718-a43d-9499f54c6f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:38.448329 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3876c9-daa4-4718-a43d-9499f54c6f98 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:38.448974 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3876c9-daa4-4718-a43d-9499f54c6f98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:38.451387 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3876c9-daa4-4718-a43d-9499f54c6f98 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:38.496851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7264646c-1b18-4ca1-a9a4-2d2e9bb5cb93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8eebf5044a64ff6b9b3d946ac2ab129'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:38.534647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1371/2707] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:38 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:38.539031 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3876c9-daa4-4718-a43d-9499f54c6f98 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:38.550384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7264646c-1b18-4ca1-a9a4-2d2e9bb5cb93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8eebf5044a64ff6b9b3d946ac2ab129'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:38.564601 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7264646c-1b18-4ca1-a9a4-2d2e9bb5cb93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8eebf5044a64ff6b9b3d946ac2ab129'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:38.565245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7264646c-1b18-4ca1-a9a4-2d2e9bb5cb93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8eebf5044a64ff6b9b3d946ac2ab129'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:38.573109 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7264646c-1b18-4ca1-a9a4-2d2e9bb5cb93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8eebf5044a64ff6b9b3d946ac2ab129'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:38.583837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1337/2708] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:38 2022] DELETE /identity/v3/users/d8eebf5044a64ff6b9b3d946ac2ab129 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:38.585338 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7562c2-2830-4675-bc6e-6c73cac76a7d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:38.585486 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7562c2-2830-4675-bc6e-6c73cac76a7d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:38.585752 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7562c2-2830-4675-bc6e-6c73cac76a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:38.607453 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ec7562c2-2830-4675-bc6e-6c73cac76a7d None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:38.608673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ec7562c2-2830-4675-bc6e-6c73cac76a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:38.609149 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3876c9-daa4-4718-a43d-9499f54c6f98 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:38.667063 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb37c0e-17df-4130-934d-6967aa21f0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:38.667589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb37c0e-17df-4130-934d-6967aa21f0f1 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:38.668530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb37c0e-17df-4130-934d-6967aa21f0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:38.668800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb37c0e-17df-4130-934d-6967aa21f0f1 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:38.669359 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1338/2709] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:38.669515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb37c0e-17df-4130-934d-6967aa21f0f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:38.671365 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb37c0e-17df-4130-934d-6967aa21f0f1 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:38.679132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7562c2-2830-4675-bc6e-6c73cac76a7d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:38.680485 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ec7562c2-2830-4675-bc6e-6c73cac76a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:38.694647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:38.738181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7562c2-2830-4675-bc6e-6c73cac76a7d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:38.742597 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2a8f2-447d-4139-a8ac-8732ae6e7f33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:38.743281 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2a8f2-447d-4139-a8ac-8732ae6e7f33 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:38.743635 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2a8f2-447d-4139-a8ac-8732ae6e7f33 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:38.744019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2a8f2-447d-4139-a8ac-8732ae6e7f33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:38.753080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2a8f2-447d-4139-a8ac-8732ae6e7f33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:38.754488 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2a8f2-447d-4139-a8ac-8732ae6e7f33 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:38.763841 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1339/2710] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:08:38.786620 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1372/2711] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:38 2022] POST /identity/v3/users => generated 291 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:38.790583 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2a8f2-447d-4139-a8ac-8732ae6e7f33 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:38.792032 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-23d2a8f2-447d-4139-a8ac-8732ae6e7f33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:38.807245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2a8f2-447d-4139-a8ac-8732ae6e7f33 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:38.811805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f9c77-6b40-4364-9333-3d3970957dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:38.812785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f9c77-6b40-4364-9333-3d3970957dba admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:38.812785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f9c77-6b40-4364-9333-3d3970957dba admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:38.813087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f9c77-6b40-4364-9333-3d3970957dba admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:38.813588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f9c77-6b40-4364-9333-3d3970957dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:38.815247 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f9c77-6b40-4364-9333-3d3970957dba admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:38.826004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1340/2712] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:38.830124 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb37c0e-17df-4130-934d-6967aa21f0f1 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:38.841838 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb37c0e-17df-4130-934d-6967aa21f0f1 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:38.847093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2e6fa-ef3b-432d-a53f-16be41eddb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:38.847707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2e6fa-ef3b-432d-a53f-16be41eddb1c admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:38.847986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2e6fa-ef3b-432d-a53f-16be41eddb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:38.848261 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2e6fa-ef3b-432d-a53f-16be41eddb1c admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:38.848963 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2e6fa-ef3b-432d-a53f-16be41eddb1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:38.850581 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2e6fa-ef3b-432d-a53f-16be41eddb1c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:38.862097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1373/2713] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:38.866476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f9c77-6b40-4364-9333-3d3970957dba admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:38.876700 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f9c77-6b40-4364-9333-3d3970957dba admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:38.881332 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657e2c3-48d8-4d57-a47d-413530a89a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:38.881988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657e2c3-48d8-4d57-a47d-413530a89a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:38.882266 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657e2c3-48d8-4d57-a47d-413530a89a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:38.882539 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657e2c3-48d8-4d57-a47d-413530a89a73 admin admin] PATH_INFO: `/v3/projects/736f37b101ed43369000a5ba9d93e90f/users/4ac5fd494df14ee1b0faf745dcc8c1e3/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:38.897581 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657e2c3-48d8-4d57-a47d-413530a89a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=736f37b101ed43369000a5ba9d93e90f, user_id=4ac5fd494df14ee1b0faf745dcc8c1e3, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:38.900284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657e2c3-48d8-4d57-a47d-413530a89a73 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:39.005564 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1341/2714] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:38 2022] PUT /identity/v3/projects/736f37b101ed43369000a5ba9d93e90f/users/4ac5fd494df14ee1b0faf745dcc8c1e3/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:39.010639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff6deb-0a28-40e8-b1e4-ba911eaae203 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:39.010972 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff6deb-0a28-40e8-b1e4-ba911eaae203 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:39.011265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff6deb-0a28-40e8-b1e4-ba911eaae203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:39.018486 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:39.032707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-4eff6deb-0a28-40e8-b1e4-ba911eaae203 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:39.033939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4eff6deb-0a28-40e8-b1e4-ba911eaae203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:39.093328 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1374/2715] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:39.103004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657e2c3-48d8-4d57-a47d-413530a89a73 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:39.162543 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657e2c3-48d8-4d57-a47d-413530a89a73 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:39.166483 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf328cff-3fa8-496f-97c1-d572b5628aeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:39.167115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf328cff-3fa8-496f-97c1-d572b5628aeb service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:39.167432 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf328cff-3fa8-496f-97c1-d572b5628aeb service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:39.167823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf328cff-3fa8-496f-97c1-d572b5628aeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:39.177271 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf328cff-3fa8-496f-97c1-d572b5628aeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:39.178649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf328cff-3fa8-496f-97c1-d572b5628aeb service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:39.188180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1342/2716] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:39.237644 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff6deb-0a28-40e8-b1e4-ba911eaae203 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:39.239138 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4eff6deb-0a28-40e8-b1e4-ba911eaae203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:39.251854 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff6deb-0a28-40e8-b1e4-ba911eaae203 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:39.257132 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9e529-4a41-44e5-a8f4-1046f2c2c460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:39.259986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9e529-4a41-44e5-a8f4-1046f2c2c460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:39.259986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9e529-4a41-44e5-a8f4-1046f2c2c460 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:39.259986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9e529-4a41-44e5-a8f4-1046f2c2c460 admin admin] PATH_INFO: `/v3/projects/a1eaab11ad3e4ed1a282204cb6941ef2` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:39.259986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9e529-4a41-44e5-a8f4-1046f2c2c460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1eaab11ad3e4ed1a282204cb6941ef2)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:39.263275 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9e529-4a41-44e5-a8f4-1046f2c2c460 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:39.407567 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-91c9e529-4a41-44e5-a8f4-1046f2c2c460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1eaab11ad3e4ed1a282204cb6941ef2', 'request_id': 'req-91c9e529-4a41-44e5-a8f4-1046f2c2c460'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:39.490359 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-91c9e529-4a41-44e5-a8f4-1046f2c2c460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1eaab11ad3e4ed1a282204cb6941ef2', 'request_id': 'req-91c9e529-4a41-44e5-a8f4-1046f2c2c460'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:39.497701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-91c9e529-4a41-44e5-a8f4-1046f2c2c460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1eaab11ad3e4ed1a282204cb6941ef2', 'request_id': 'req-91c9e529-4a41-44e5-a8f4-1046f2c2c460'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:39.512461 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-91c9e529-4a41-44e5-a8f4-1046f2c2c460 admin admin] The token cache is being invalidate because project a1eaab11ad3e4ed1a282204cb6941ef2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:39.512461 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-91c9e529-4a41-44e5-a8f4-1046f2c2c460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:39.513082 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-91c9e529-4a41-44e5-a8f4-1046f2c2c460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:39.520652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1375/2717] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:39 2022] DELETE /identity/v3/projects/a1eaab11ad3e4ed1a282204cb6941ef2 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:39.520865 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf328cff-3fa8-496f-97c1-d572b5628aeb service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:39.530291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf328cff-3fa8-496f-97c1-d572b5628aeb service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:39.594105 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa231e65-06f1-47aa-aa7f-7089cbdff7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:39.594808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa231e65-06f1-47aa-aa7f-7089cbdff7fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:39.595276 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa231e65-06f1-47aa-aa7f-7089cbdff7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:39.595612 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa231e65-06f1-47aa-aa7f-7089cbdff7fc admin admin] PATH_INFO: `/v3/users/c758a87c114b4b90a4eacc5af1081e1c` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:39.597045 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa231e65-06f1-47aa-aa7f-7089cbdff7fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c758a87c114b4b90a4eacc5af1081e1c)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:39.599213 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa231e65-06f1-47aa-aa7f-7089cbdff7fc admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:39.712738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:39.823282 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fa231e65-06f1-47aa-aa7f-7089cbdff7fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c758a87c114b4b90a4eacc5af1081e1c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:39.916302 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fa231e65-06f1-47aa-aa7f-7089cbdff7fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c758a87c114b4b90a4eacc5af1081e1c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:39.965108 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fa231e65-06f1-47aa-aa7f-7089cbdff7fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c758a87c114b4b90a4eacc5af1081e1c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:39.965944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fa231e65-06f1-47aa-aa7f-7089cbdff7fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c758a87c114b4b90a4eacc5af1081e1c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:40.000652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fa231e65-06f1-47aa-aa7f-7089cbdff7fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c758a87c114b4b90a4eacc5af1081e1c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:40.015245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1343/2718] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:39 2022] DELETE /identity/v3/users/c758a87c114b4b90a4eacc5af1081e1c => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:40.724774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9e529-4a41-44e5-a8f4-1046f2c2c460 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:40.726229 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-91c9e529-4a41-44e5-a8f4-1046f2c2c460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:40.739712 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9e529-4a41-44e5-a8f4-1046f2c2c460 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:40.805459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f1faa-9205-4e07-8406-1ba8714b99d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:40.806164 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f1faa-9205-4e07-8406-1ba8714b99d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:40.806411 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f1faa-9205-4e07-8406-1ba8714b99d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:40.806690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f1faa-9205-4e07-8406-1ba8714b99d3 admin admin] PATH_INFO: `/v3/projects/0527e71d814d4ba08ce0504bc06a073f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:40.807877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f1faa-9205-4e07-8406-1ba8714b99d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0527e71d814d4ba08ce0504bc06a073f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:40.809820 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f1faa-9205-4e07-8406-1ba8714b99d3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:40.827370 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583ecd2-02d5-4a4b-93d7-3277d38ceb07 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:40.827783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583ecd2-02d5-4a4b-93d7-3277d38ceb07 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:40.827783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583ecd2-02d5-4a4b-93d7-3277d38ceb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:40.872874 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-3583ecd2-02d5-4a4b-93d7-3277d38ceb07 None None] MFA Rules not processed for user `4ac5fd494df14ee1b0faf745dcc8c1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:40.875825 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3583ecd2-02d5-4a4b-93d7-3277d38ceb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:40.925287 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:40.952521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f74f1faa-9205-4e07-8406-1ba8714b99d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0527e71d814d4ba08ce0504bc06a073f', 'request_id': 'req-f74f1faa-9205-4e07-8406-1ba8714b99d3'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:40.994081 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1344/2719] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:41.000459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583ecd2-02d5-4a4b-93d7-3277d38ceb07 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:41.002736 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3583ecd2-02d5-4a4b-93d7-3277d38ceb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:41.039648 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f74f1faa-9205-4e07-8406-1ba8714b99d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0527e71d814d4ba08ce0504bc06a073f', 'request_id': 'req-f74f1faa-9205-4e07-8406-1ba8714b99d3'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:41.049981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f74f1faa-9205-4e07-8406-1ba8714b99d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0527e71d814d4ba08ce0504bc06a073f', 'request_id': 'req-f74f1faa-9205-4e07-8406-1ba8714b99d3'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:41.064761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f74f1faa-9205-4e07-8406-1ba8714b99d3 admin admin] The token cache is being invalidate because project 0527e71d814d4ba08ce0504bc06a073f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:41.065081 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f74f1faa-9205-4e07-8406-1ba8714b99d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:41.065729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f74f1faa-9205-4e07-8406-1ba8714b99d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:41.067840 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1376/2720] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:40 2022] DELETE /identity/v3/projects/0527e71d814d4ba08ce0504bc06a073f => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:41.071750 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f1faa-9205-4e07-8406-1ba8714b99d3 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:41.073408 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f74f1faa-9205-4e07-8406-1ba8714b99d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:41.091001 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583ecd2-02d5-4a4b-93d7-3277d38ceb07 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:41.150509 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f1faa-9205-4e07-8406-1ba8714b99d3 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:41.155233 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:41.157179 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc27d0-f889-4597-952e-406174853ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:41.157867 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc27d0-f889-4597-952e-406174853ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:41.158285 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc27d0-f889-4597-952e-406174853ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:41.158562 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc27d0-f889-4597-952e-406174853ec1 admin admin] PATH_INFO: `/v3/users/e5a2b063e0f94f19aea009fc12859e66` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:41.159817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc27d0-f889-4597-952e-406174853ec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5a2b063e0f94f19aea009fc12859e66)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:41.160531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f01a0c-2c92-4c25-8308-ac851e1b5129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:41.161201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f01a0c-2c92-4c25-8308-ac851e1b5129 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:41.161543 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f01a0c-2c92-4c25-8308-ac851e1b5129 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:41.161879 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f01a0c-2c92-4c25-8308-ac851e1b5129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:41.163742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc27d0-f889-4597-952e-406174853ec1 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:41.164267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-46f01a0c-2c92-4c25-8308-ac851e1b5129 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:41.243289 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f01a0c-2c92-4c25-8308-ac851e1b5129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:41.244848 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f01a0c-2c92-4c25-8308-ac851e1b5129 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:41.256188 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1345/2721] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:08:41.390072 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-75bc27d0-f889-4597-952e-406174853ec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5a2b063e0f94f19aea009fc12859e66'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:41.473381 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-75bc27d0-f889-4597-952e-406174853ec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5a2b063e0f94f19aea009fc12859e66'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:41.474246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-75bc27d0-f889-4597-952e-406174853ec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5a2b063e0f94f19aea009fc12859e66'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:41.487731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-75bc27d0-f889-4597-952e-406174853ec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5a2b063e0f94f19aea009fc12859e66'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:41.505986 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-75bc27d0-f889-4597-952e-406174853ec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5a2b063e0f94f19aea009fc12859e66'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:41.513093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1377/2722] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:41 2022] DELETE /identity/v3/users/e5a2b063e0f94f19aea009fc12859e66 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:41.992456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f01a0c-2c92-4c25-8308-ac851e1b5129 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:42.006785 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f01a0c-2c92-4c25-8308-ac851e1b5129 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:42.079145 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e11b1-5e5f-426d-99df-6d5fa93b020b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:42.079900 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e11b1-5e5f-426d-99df-6d5fa93b020b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:42.080227 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e11b1-5e5f-426d-99df-6d5fa93b020b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:42.080426 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e11b1-5e5f-426d-99df-6d5fa93b020b admin admin] PATH_INFO: `/v3/projects/8341a94475e9440680b587e43023724f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:42.081563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e11b1-5e5f-426d-99df-6d5fa93b020b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8341a94475e9440680b587e43023724f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:42.083535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e11b1-5e5f-426d-99df-6d5fa93b020b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:42.210327 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1f4e11b1-5e5f-426d-99df-6d5fa93b020b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8341a94475e9440680b587e43023724f', 'request_id': 'req-1f4e11b1-5e5f-426d-99df-6d5fa93b020b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:42.216419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1f4e11b1-5e5f-426d-99df-6d5fa93b020b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8341a94475e9440680b587e43023724f', 'request_id': 'req-1f4e11b1-5e5f-426d-99df-6d5fa93b020b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:42.222472 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1f4e11b1-5e5f-426d-99df-6d5fa93b020b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8341a94475e9440680b587e43023724f', 'request_id': 'req-1f4e11b1-5e5f-426d-99df-6d5fa93b020b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:42.324197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1f4e11b1-5e5f-426d-99df-6d5fa93b020b admin admin] The token cache is being invalidate because project 8341a94475e9440680b587e43023724f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:42.324562 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1f4e11b1-5e5f-426d-99df-6d5fa93b020b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:42.325247 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1f4e11b1-5e5f-426d-99df-6d5fa93b020b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:42.326853 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1346/2723] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:41 2022] DELETE /identity/v3/projects/8341a94475e9440680b587e43023724f => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:42.331782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51961b2-4cf5-4084-af25-3c3ea32cdf1e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:42.332310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51961b2-4cf5-4084-af25-3c3ea32cdf1e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:42.332487 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51961b2-4cf5-4084-af25-3c3ea32cdf1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:42.355736 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e51961b2-4cf5-4084-af25-3c3ea32cdf1e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:42.357135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e51961b2-4cf5-4084-af25-3c3ea32cdf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:42.393655 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:42.424713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1378/2724] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:42.432535 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1362971-947c-4e23-a66b-dc39685d6127 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:42.432818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1362971-947c-4e23-a66b-dc39685d6127 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:42.433100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1362971-947c-4e23-a66b-dc39685d6127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:42.455059 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b1362971-947c-4e23-a66b-dc39685d6127 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:42.456314 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b1362971-947c-4e23-a66b-dc39685d6127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:42.464427 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1347/2725] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:42.469407 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51961b2-4cf5-4084-af25-3c3ea32cdf1e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:42.479812 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51961b2-4cf5-4084-af25-3c3ea32cdf1e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:42.485653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e536b-2851-4466-9e64-c660a27fce2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:42.486688 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e536b-2851-4466-9e64-c660a27fce2a admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:42.487008 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e536b-2851-4466-9e64-c660a27fce2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:42.487283 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e536b-2851-4466-9e64-c660a27fce2a admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:42.487959 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e536b-2851-4466-9e64-c660a27fce2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:08:42.488273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e536b-2851-4466-9e64-c660a27fce2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:42.489871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e536b-2851-4466-9e64-c660a27fce2a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:42.502970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1379/2726] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:08:42.507512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1362971-947c-4e23-a66b-dc39685d6127 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:42.517500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1362971-947c-4e23-a66b-dc39685d6127 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:42.522658 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0484be4-73f6-43de-a06e-3d83b72bfcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:42.523886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0484be4-73f6-43de-a06e-3d83b72bfcee admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:42.523886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0484be4-73f6-43de-a06e-3d83b72bfcee admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:42.523886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0484be4-73f6-43de-a06e-3d83b72bfcee admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:42.524450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0484be4-73f6-43de-a06e-3d83b72bfcee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:42.526340 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0484be4-73f6-43de-a06e-3d83b72bfcee admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:42.638034 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1348/2727] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:42 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:42.642563 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e536b-2851-4466-9e64-c660a27fce2a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:42.653269 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e536b-2851-4466-9e64-c660a27fce2a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:42.710446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388d733-d600-4444-bb43-9966a3b7f340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:42.711080 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388d733-d600-4444-bb43-9966a3b7f340 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:42.711361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388d733-d600-4444-bb43-9966a3b7f340 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:42.711635 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388d733-d600-4444-bb43-9966a3b7f340 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:42.712267 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388d733-d600-4444-bb43-9966a3b7f340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:42.714161 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4388d733-d600-4444-bb43-9966a3b7f340 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:42.810944 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1380/2728] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:42 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:42.815056 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0484be4-73f6-43de-a06e-3d83b72bfcee admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:42.821675 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:42.825990 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0484be4-73f6-43de-a06e-3d83b72bfcee admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:42.830712 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166cb00-b3f7-48dd-a015-e944f64a8b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:42.831351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166cb00-b3f7-48dd-a015-e944f64a8b27 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:42.831628 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166cb00-b3f7-48dd-a015-e944f64a8b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:42.831895 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166cb00-b3f7-48dd-a015-e944f64a8b27 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:42.832527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166cb00-b3f7-48dd-a015-e944f64a8b27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:42.834120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166cb00-b3f7-48dd-a015-e944f64a8b27 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:42.845197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1349/2729] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:42.849697 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388d733-d600-4444-bb43-9966a3b7f340 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:42.860097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4388d733-d600-4444-bb43-9966a3b7f340 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:42.864837 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89761d-505b-4d13-b587-6828ac4ba7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:42.865427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89761d-505b-4d13-b587-6828ac4ba7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:42.865707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89761d-505b-4d13-b587-6828ac4ba7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:42.866033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89761d-505b-4d13-b587-6828ac4ba7f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:42.866669 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a89761d-505b-4d13-b587-6828ac4ba7f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:42.868306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a89761d-505b-4d13-b587-6828ac4ba7f6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:42.878954 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1381/2730] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:42.883283 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166cb00-b3f7-48dd-a015-e944f64a8b27 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:42.893850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166cb00-b3f7-48dd-a015-e944f64a8b27 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:42.898769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb222f2-03bd-45aa-ab49-befbb3c3adb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:42.899624 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb222f2-03bd-45aa-ab49-befbb3c3adb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:42.899911 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb222f2-03bd-45aa-ab49-befbb3c3adb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:42.900183 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb222f2-03bd-45aa-ab49-befbb3c3adb4 admin admin] PATH_INFO: `/v3/projects/c1aa7132baef46debf5f9f5e6f2885ef/users/7e6e5ac0689b47aea4e79192454fd04a/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:42.916325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb222f2-03bd-45aa-ab49-befbb3c3adb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1aa7132baef46debf5f9f5e6f2885ef, user_id=7e6e5ac0689b47aea4e79192454fd04a, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:42.919118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb222f2-03bd-45aa-ab49-befbb3c3adb4 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:43.028229 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1350/2731] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:42 2022] PUT /identity/v3/projects/c1aa7132baef46debf5f9f5e6f2885ef/users/7e6e5ac0689b47aea4e79192454fd04a/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:43.045672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a9774-3b29-4982-b37b-beed108a8d04 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:43.045672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a9774-3b29-4982-b37b-beed108a8d04 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:43.045672 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a9774-3b29-4982-b37b-beed108a8d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:43.076244 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-dd3a9774-3b29-4982-b37b-beed108a8d04 None None] MFA Rules not processed for user `7e6e5ac0689b47aea4e79192454fd04a`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:43.076244 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-dd3a9774-3b29-4982-b37b-beed108a8d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:43.118588 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a9c1a-b1df-4f89-826d-7006f446bcd7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:43.118868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a9c1a-b1df-4f89-826d-7006f446bcd7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:43.119243 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a9c1a-b1df-4f89-826d-7006f446bcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:43.139997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9f2a9c1a-b1df-4f89-826d-7006f446bcd7 None None] MFA Rules not processed for user `4ac5fd494df14ee1b0faf745dcc8c1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:43.141224 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9f2a9c1a-b1df-4f89-826d-7006f446bcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:43.144657 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1382/2732] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:43.149781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a9774-3b29-4982-b37b-beed108a8d04 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:43.185336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:43.206157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1351/2733] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:43.215701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a9774-3b29-4982-b37b-beed108a8d04 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:43.220611 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e8c0e-123c-476a-bd39-4c44a88c20cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:43.222997 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e8c0e-123c-476a-bd39-4c44a88c20cd admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:43.222997 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e8c0e-123c-476a-bd39-4c44a88c20cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:43.222997 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e8c0e-123c-476a-bd39-4c44a88c20cd admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:43.222997 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e8c0e-123c-476a-bd39-4c44a88c20cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:43.224698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e8c0e-123c-476a-bd39-4c44a88c20cd admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:43.409939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1383/2734] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:43 2022] POST /identity/v3/projects => generated 385 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:43.411322 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a9c1a-b1df-4f89-826d-7006f446bcd7 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:43.426136 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a9c1a-b1df-4f89-826d-7006f446bcd7 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:43.501400 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62472120-1384-43ad-b5bd-b477a1599775 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:43.501720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62472120-1384-43ad-b5bd-b477a1599775 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:43.501954 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62472120-1384-43ad-b5bd-b477a1599775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:43.529230 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-62472120-1384-43ad-b5bd-b477a1599775 None None] MFA Rules not processed for user `46cce59255af494eaa238ab8c8cddc51`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:43.530482 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-62472120-1384-43ad-b5bd-b477a1599775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:43.558048 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb03a40-d37f-4dd6-ad49-a0b554997302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:43.559028 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb03a40-d37f-4dd6-ad49-a0b554997302 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:43.559668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb03a40-d37f-4dd6-ad49-a0b554997302 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:43.560145 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb03a40-d37f-4dd6-ad49-a0b554997302 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:43.561234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb03a40-d37f-4dd6-ad49-a0b554997302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:43.565350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb03a40-d37f-4dd6-ad49-a0b554997302 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:43.608386 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1384/2735] 10.222.0.91 () {58 vars in 967 bytes} [Thu Sep 8 14:08:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:43.623466 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62472120-1384-43ad-b5bd-b477a1599775 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:43.625067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-62472120-1384-43ad-b5bd-b477a1599775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:43.686593 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1352/2736] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:43 2022] POST /identity/v3/users => generated 293 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:43.691753 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb03a40-d37f-4dd6-ad49-a0b554997302 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:43.703994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb03a40-d37f-4dd6-ad49-a0b554997302 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:43.709237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdaf64-de22-483b-9509-a19c4fe33da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:43.709908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdaf64-de22-483b-9509-a19c4fe33da3 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:43.710185 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdaf64-de22-483b-9509-a19c4fe33da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:43.710521 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdaf64-de22-483b-9509-a19c4fe33da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:43.711254 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfdaf64-de22-483b-9509-a19c4fe33da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:43.712943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfdaf64-de22-483b-9509-a19c4fe33da3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:43.713489 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62472120-1384-43ad-b5bd-b477a1599775 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:43.722654 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537f223-321c-4be3-a895-a5ad1ded81de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b3b5ae758fe847adaae0edaa0be095c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:43.723205 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537f223-321c-4be3-a895-a5ad1ded81de service swift] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:43.723482 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537f223-321c-4be3-a895-a5ad1ded81de service swift] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:43.723841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537f223-321c-4be3-a895-a5ad1ded81de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:43.726121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1353/2737] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:43.730842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdaf64-de22-483b-9509-a19c4fe33da3 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:43.739626 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537f223-321c-4be3-a895-a5ad1ded81de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:43.741058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdaf64-de22-483b-9509-a19c4fe33da3 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:43.741477 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537f223-321c-4be3-a895-a5ad1ded81de service swift] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:43.746191 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888d527-77ae-4593-ae67-b6f716f87199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:43.747649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888d527-77ae-4593-ae67-b6f716f87199 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:43.747943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888d527-77ae-4593-ae67-b6f716f87199 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:43.748288 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888d527-77ae-4593-ae67-b6f716f87199 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:43.748991 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888d527-77ae-4593-ae67-b6f716f87199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:43.750721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888d527-77ae-4593-ae67-b6f716f87199 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:43.752683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1385/2738] 10.222.0.91 () {60 vars in 1384 bytes} [Thu Sep 8 14:08:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 14:08:43.763932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1354/2739] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:43.768035 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537f223-321c-4be3-a895-a5ad1ded81de service swift] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:43.781156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537f223-321c-4be3-a895-a5ad1ded81de service swift] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:43.785761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814009f4-d2aa-47e5-9918-47c3b2e9ffa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:43.786479 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814009f4-d2aa-47e5-9918-47c3b2e9ffa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:43.786856 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814009f4-d2aa-47e5-9918-47c3b2e9ffa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:43.787226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814009f4-d2aa-47e5-9918-47c3b2e9ffa9 admin admin] PATH_INFO: `/v3/projects/6ca780589ff14de98c17aa08d581c2e7/users/124fa9afc19f41ddb6d1989b19d550f9/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:43.802452 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814009f4-d2aa-47e5-9918-47c3b2e9ffa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca780589ff14de98c17aa08d581c2e7, user_id=124fa9afc19f41ddb6d1989b19d550f9, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:43.805126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814009f4-d2aa-47e5-9918-47c3b2e9ffa9 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:43.942371 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1386/2740] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:43 2022] PUT /identity/v3/projects/6ca780589ff14de98c17aa08d581c2e7/users/124fa9afc19f41ddb6d1989b19d550f9/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:43.953038 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff91649-2658-47ec-9702-c4f83920efc1 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:43.953374 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff91649-2658-47ec-9702-c4f83920efc1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:43.953715 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff91649-2658-47ec-9702-c4f83920efc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:43.992616 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-5ff91649-2658-47ec-9702-c4f83920efc1 None None] MFA Rules not processed for user `124fa9afc19f41ddb6d1989b19d550f9`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:43.995522 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5ff91649-2658-47ec-9702-c4f83920efc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:44.033125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:44.081955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1355/2741] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:44.086102 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814009f4-d2aa-47e5-9918-47c3b2e9ffa9 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:44.116402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:44.151493 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814009f4-d2aa-47e5-9918-47c3b2e9ffa9 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:44.156361 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dd199-f335-4c42-b9e2-6fabd4c74a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:44.157014 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dd199-f335-4c42-b9e2-6fabd4c74a15 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:44.157766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dd199-f335-4c42-b9e2-6fabd4c74a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:44.157766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7dd199-f335-4c42-b9e2-6fabd4c74a15 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:44.158195 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7dd199-f335-4c42-b9e2-6fabd4c74a15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:44.160157 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7dd199-f335-4c42-b9e2-6fabd4c74a15 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:44.253073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1387/2742] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:44 2022] POST /identity/v3/projects => generated 383 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:44.257215 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff91649-2658-47ec-9702-c4f83920efc1 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:44.269958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff91649-2658-47ec-9702-c4f83920efc1 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:44.331190 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b86d9-0af7-4435-a86c-35c514db15f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:44.331807 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b86d9-0af7-4435-a86c-35c514db15f0 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:44.332087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b86d9-0af7-4435-a86c-35c514db15f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:44.332365 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b86d9-0af7-4435-a86c-35c514db15f0 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:44.333009 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b86d9-0af7-4435-a86c-35c514db15f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:44.334978 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b86d9-0af7-4435-a86c-35c514db15f0 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:44.442606 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1356/2743] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:44 2022] POST /identity/v3/users => generated 298 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:44.453373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dd199-f335-4c42-b9e2-6fabd4c74a15 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:44.464000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7dd199-f335-4c42-b9e2-6fabd4c74a15 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:44.469632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114e16d-bc58-4664-b1a7-c0d9f7643a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:44.470351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114e16d-bc58-4664-b1a7-c0d9f7643a36 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:44.470730 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114e16d-bc58-4664-b1a7-c0d9f7643a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:44.471246 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114e16d-bc58-4664-b1a7-c0d9f7643a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:44.474368 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114e16d-bc58-4664-b1a7-c0d9f7643a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:44.476175 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114e16d-bc58-4664-b1a7-c0d9f7643a36 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:44.490306 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1388/2744] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:44.491842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b86d9-0af7-4435-a86c-35c514db15f0 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:44.503721 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b86d9-0af7-4435-a86c-35c514db15f0 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:44.514376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114e16d-bc58-4664-b1a7-c0d9f7643a36 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:44.517143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b114e16d-bc58-4664-b1a7-c0d9f7643a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:44.520703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927c9f5-0a0e-4e5d-b1a4-3fa427226028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:44.521292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927c9f5-0a0e-4e5d-b1a4-3fa427226028 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:44.521823 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927c9f5-0a0e-4e5d-b1a4-3fa427226028 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:44.522103 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927c9f5-0a0e-4e5d-b1a4-3fa427226028 admin admin] PATH_INFO: `/v3/projects/4bc79e09daee475fa44b1d20cf86896d/users/badb1f2fb7ca4c6ea5c8be29a2b347c5/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:44.531405 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114e16d-bc58-4664-b1a7-c0d9f7643a36 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:44.536547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd8cb16-c4b2-4de5-b043-c5c45bc1ebc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:44.537414 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8cb16-c4b2-4de5-b043-c5c45bc1ebc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:44.538081 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8cb16-c4b2-4de5-b043-c5c45bc1ebc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:44.538364 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd8cb16-c4b2-4de5-b043-c5c45bc1ebc3 admin admin] PATH_INFO: `/v3/users/206c96a9fff349bdbe63ff045cfe74cf` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:44.539597 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8cb16-c4b2-4de5-b043-c5c45bc1ebc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=206c96a9fff349bdbe63ff045cfe74cf)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:44.540978 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927c9f5-0a0e-4e5d-b1a4-3fa427226028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc79e09daee475fa44b1d20cf86896d, user_id=badb1f2fb7ca4c6ea5c8be29a2b347c5, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:44.541546 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd8cb16-c4b2-4de5-b043-c5c45bc1ebc3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:44.543738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927c9f5-0a0e-4e5d-b1a4-3fa427226028 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:44.628570 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1357/2745] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:44 2022] PUT /identity/v3/projects/4bc79e09daee475fa44b1d20cf86896d/users/badb1f2fb7ca4c6ea5c8be29a2b347c5/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:44.632587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927c9f5-0a0e-4e5d-b1a4-3fa427226028 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:44.789958 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1dd8cb16-c4b2-4de5-b043-c5c45bc1ebc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '206c96a9fff349bdbe63ff045cfe74cf'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:44.819567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927c9f5-0a0e-4e5d-b1a4-3fa427226028 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:44.824849 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e952d7d-332a-465c-9b51-94717b040bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:44.825474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e952d7d-332a-465c-9b51-94717b040bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:44.825751 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e952d7d-332a-465c-9b51-94717b040bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:44.826021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e952d7d-332a-465c-9b51-94717b040bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:44.826657 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e952d7d-332a-465c-9b51-94717b040bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:44.828321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e952d7d-332a-465c-9b51-94717b040bd9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:44.838411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1358/2746] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:08:44.843062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e952d7d-332a-465c-9b51-94717b040bd9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:44.853694 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e952d7d-332a-465c-9b51-94717b040bd9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:44.859018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e306d57-0524-441a-b8d3-c43a151d3686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:44.859661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e306d57-0524-441a-b8d3-c43a151d3686 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:44.859933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e306d57-0524-441a-b8d3-c43a151d3686 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:44.860200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e306d57-0524-441a-b8d3-c43a151d3686 admin admin] PATH_INFO: `/v3/projects/4bc79e09daee475fa44b1d20cf86896d/users/badb1f2fb7ca4c6ea5c8be29a2b347c5/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:44.862080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e306d57-0524-441a-b8d3-c43a151d3686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc79e09daee475fa44b1d20cf86896d, user_id=badb1f2fb7ca4c6ea5c8be29a2b347c5, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:44.864713 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e306d57-0524-441a-b8d3-c43a151d3686 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:44.870457 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1dd8cb16-c4b2-4de5-b043-c5c45bc1ebc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '206c96a9fff349bdbe63ff045cfe74cf'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:44.871344 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1dd8cb16-c4b2-4de5-b043-c5c45bc1ebc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '206c96a9fff349bdbe63ff045cfe74cf'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:44.878743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1359/2747] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:44 2022] PUT /identity/v3/projects/4bc79e09daee475fa44b1d20cf86896d/users/badb1f2fb7ca4c6ea5c8be29a2b347c5/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:44.884057 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1dd8cb16-c4b2-4de5-b043-c5c45bc1ebc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '206c96a9fff349bdbe63ff045cfe74cf'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:44.891144 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a881f0-63d1-42a7-8284-be8380ea8426 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:44.891544 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a881f0-63d1-42a7-8284-be8380ea8426 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:44.891920 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a881f0-63d1-42a7-8284-be8380ea8426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:44.900849 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1dd8cb16-c4b2-4de5-b043-c5c45bc1ebc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '206c96a9fff349bdbe63ff045cfe74cf'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:44.909348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1389/2748] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:44 2022] DELETE /identity/v3/users/206c96a9fff349bdbe63ff045cfe74cf => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:44.914133 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f17a5-0aba-4aea-aa1f-2249f6d167b4 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:44.914428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f17a5-0aba-4aea-aa1f-2249f6d167b4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:44.914700 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f17a5-0aba-4aea-aa1f-2249f6d167b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:44.923452 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c8a881f0-63d1-42a7-8284-be8380ea8426 None None] MFA Rules not processed for user `badb1f2fb7ca4c6ea5c8be29a2b347c5`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:44.925669 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c8a881f0-63d1-42a7-8284-be8380ea8426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:44.936188 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:44.936649 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-103f17a5-0aba-4aea-aa1f-2249f6d167b4 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:44.937990 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-103f17a5-0aba-4aea-aa1f-2249f6d167b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:44.999580 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1390/2749] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:45.002496 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1360/2750] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:45.010083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f17a5-0aba-4aea-aa1f-2249f6d167b4 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:45.011592 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-103f17a5-0aba-4aea-aa1f-2249f6d167b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:45.013582 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a881f0-63d1-42a7-8284-be8380ea8426 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:45.015205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c8a881f0-63d1-42a7-8284-be8380ea8426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:45.032978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:45.076990 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a881f0-63d1-42a7-8284-be8380ea8426 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:45.080741 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0030f-b81c-4641-a58e-007aa757ad74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:45.081442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0030f-b81c-4641-a58e-007aa757ad74 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:45.081775 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0030f-b81c-4641-a58e-007aa757ad74 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:45.082212 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0030f-b81c-4641-a58e-007aa757ad74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:45.084145 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9db0030f-b81c-4641-a58e-007aa757ad74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:45.086077 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f17a5-0aba-4aea-aa1f-2249f6d167b4 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:45.090273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d2d26-0c6c-4ddb-9db4-8a15807ab4d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:45.091435 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d2d26-0c6c-4ddb-9db4-8a15807ab4d2 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:45.091946 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d2d26-0c6c-4ddb-9db4-8a15807ab4d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:45.092193 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d2d26-0c6c-4ddb-9db4-8a15807ab4d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:45.106493 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d2d26-0c6c-4ddb-9db4-8a15807ab4d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:45.108803 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d2d26-0c6c-4ddb-9db4-8a15807ab4d2 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:45.123388 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1391/2751] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:45.174051 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0030f-b81c-4641-a58e-007aa757ad74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:45.175615 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0030f-b81c-4641-a58e-007aa757ad74 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:45.186083 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1361/2752] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:08:45.613693 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d2d26-0c6c-4ddb-9db4-8a15807ab4d2 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:45.616085 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-236d2d26-0c6c-4ddb-9db4-8a15807ab4d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:45.632573 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d2d26-0c6c-4ddb-9db4-8a15807ab4d2 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:45.637603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90a25f-010b-439c-9a59-988032d0c435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:45.643781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90a25f-010b-439c-9a59-988032d0c435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:45.643781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90a25f-010b-439c-9a59-988032d0c435 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:45.643781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90a25f-010b-439c-9a59-988032d0c435 admin admin] PATH_INFO: `/v3/projects/ea4a774193494f7eab5047a98a285a7c` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:45.643781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90a25f-010b-439c-9a59-988032d0c435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea4a774193494f7eab5047a98a285a7c)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:45.643781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90a25f-010b-439c-9a59-988032d0c435 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:45.841410 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-bc90a25f-010b-439c-9a59-988032d0c435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea4a774193494f7eab5047a98a285a7c', 'request_id': 'req-bc90a25f-010b-439c-9a59-988032d0c435'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:45.930642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-bc90a25f-010b-439c-9a59-988032d0c435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea4a774193494f7eab5047a98a285a7c', 'request_id': 'req-bc90a25f-010b-439c-9a59-988032d0c435'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:45.939475 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-bc90a25f-010b-439c-9a59-988032d0c435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea4a774193494f7eab5047a98a285a7c', 'request_id': 'req-bc90a25f-010b-439c-9a59-988032d0c435'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:45.950970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-bc90a25f-010b-439c-9a59-988032d0c435 admin admin] The token cache is being invalidate because project ea4a774193494f7eab5047a98a285a7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:45.951260 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-bc90a25f-010b-439c-9a59-988032d0c435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:45.951916 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-bc90a25f-010b-439c-9a59-988032d0c435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:45.953394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1392/2753] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:45 2022] DELETE /identity/v3/projects/ea4a774193494f7eab5047a98a285a7c => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:45.961672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d90e6-91fd-4960-9a90-1a085bd266ab None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:45.962210 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d90e6-91fd-4960-9a90-1a085bd266ab None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:45.962663 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d90e6-91fd-4960-9a90-1a085bd266ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:45.986851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-dd2d90e6-91fd-4960-9a90-1a085bd266ab None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:45.986851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-dd2d90e6-91fd-4960-9a90-1a085bd266ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:46.044938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:46.053735 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1362/2754] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:46.062257 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ad1ce-26e8-4a16-8698-7a3d44288272 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:46.062544 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ad1ce-26e8-4a16-8698-7a3d44288272 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:46.062826 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ad1ce-26e8-4a16-8698-7a3d44288272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:46.083772 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-993ad1ce-26e8-4a16-8698-7a3d44288272 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:46.085017 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-993ad1ce-26e8-4a16-8698-7a3d44288272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:46.097119 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1393/2755] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:46.098355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d90e6-91fd-4960-9a90-1a085bd266ab None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:46.110757 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d90e6-91fd-4960-9a90-1a085bd266ab None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:46.115683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e4c4e-1b59-462c-aeab-ad8ae169f84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:46.117995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e4c4e-1b59-462c-aeab-ad8ae169f84e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:46.117995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e4c4e-1b59-462c-aeab-ad8ae169f84e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:46.117995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e4c4e-1b59-462c-aeab-ad8ae169f84e admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:46.117995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e4c4e-1b59-462c-aeab-ad8ae169f84e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:08:46.117995 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e4c4e-1b59-462c-aeab-ad8ae169f84e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:46.119459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e4c4e-1b59-462c-aeab-ad8ae169f84e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:46.134927 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1363/2756] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:08:46.137653 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ad1ce-26e8-4a16-8698-7a3d44288272 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:46.147469 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ad1ce-26e8-4a16-8698-7a3d44288272 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:46.153311 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa80df-be61-44a7-89a7-7ad1956a4d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:46.154334 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa80df-be61-44a7-89a7-7ad1956a4d36 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:46.154334 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa80df-be61-44a7-89a7-7ad1956a4d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:46.154526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa80df-be61-44a7-89a7-7ad1956a4d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:46.155093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa80df-be61-44a7-89a7-7ad1956a4d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:46.157105 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa80df-be61-44a7-89a7-7ad1956a4d36 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:46.279663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1394/2757] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:46 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:46.283379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e4c4e-1b59-462c-aeab-ad8ae169f84e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:46.293356 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e4c4e-1b59-462c-aeab-ad8ae169f84e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:46.358563 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f7166-763c-4736-a186-e4d5cb35f98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:46.359212 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f7166-763c-4736-a186-e4d5cb35f98a admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:46.359492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f7166-763c-4736-a186-e4d5cb35f98a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:46.359862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f7166-763c-4736-a186-e4d5cb35f98a admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:46.360376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f7166-763c-4736-a186-e4d5cb35f98a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:46.362272 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f7166-763c-4736-a186-e4d5cb35f98a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:46.437549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1364/2758] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:46 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:46.447398 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa80df-be61-44a7-89a7-7ad1956a4d36 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:46.459226 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa80df-be61-44a7-89a7-7ad1956a4d36 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:46.463939 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce9fcf-b93a-4aa7-b1bf-9f4a88301bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:46.465080 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce9fcf-b93a-4aa7-b1bf-9f4a88301bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:46.465310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce9fcf-b93a-4aa7-b1bf-9f4a88301bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:46.465598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce9fcf-b93a-4aa7-b1bf-9f4a88301bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:46.466411 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce9fcf-b93a-4aa7-b1bf-9f4a88301bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:46.468025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce9fcf-b93a-4aa7-b1bf-9f4a88301bd4 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:46.478275 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1395/2759] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:46.483895 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f7166-763c-4736-a186-e4d5cb35f98a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:46.494058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f7166-763c-4736-a186-e4d5cb35f98a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:46.499639 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d244f-dc1f-4466-93c3-a3cc6c5fc963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:46.500250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d244f-dc1f-4466-93c3-a3cc6c5fc963 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:46.500580 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d244f-dc1f-4466-93c3-a3cc6c5fc963 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:46.500858 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d244f-dc1f-4466-93c3-a3cc6c5fc963 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:46.501497 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d244f-dc1f-4466-93c3-a3cc6c5fc963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:46.504298 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d244f-dc1f-4466-93c3-a3cc6c5fc963 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:46.515038 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1365/2760] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:46.519554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce9fcf-b93a-4aa7-b1bf-9f4a88301bd4 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:46.531799 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce9fcf-b93a-4aa7-b1bf-9f4a88301bd4 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:46.539805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17df377-7edb-4328-b57f-b513919e7a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:46.539805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17df377-7edb-4328-b57f-b513919e7a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:46.539805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17df377-7edb-4328-b57f-b513919e7a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:46.539805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17df377-7edb-4328-b57f-b513919e7a75 admin admin] PATH_INFO: `/v3/projects/69ae1b2f4e9341f69d5db6b0a4425c25/users/3427af5d220c4c3c9c5a7bdb913f54f0/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:46.557128 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17df377-7edb-4328-b57f-b513919e7a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69ae1b2f4e9341f69d5db6b0a4425c25, user_id=3427af5d220c4c3c9c5a7bdb913f54f0, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:46.560093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17df377-7edb-4328-b57f-b513919e7a75 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:46.650395 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1396/2761] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:46 2022] PUT /identity/v3/projects/69ae1b2f4e9341f69d5db6b0a4425c25/users/3427af5d220c4c3c9c5a7bdb913f54f0/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:46.663460 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e9ba7-ad1c-40d3-bb78-3127660fcf48 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:46.663744 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e9ba7-ad1c-40d3-bb78-3127660fcf48 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:46.664021 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e9ba7-ad1c-40d3-bb78-3127660fcf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:46.701977 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-910e9ba7-ad1c-40d3-bb78-3127660fcf48 None None] MFA Rules not processed for user `3427af5d220c4c3c9c5a7bdb913f54f0`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:46.703675 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-910e9ba7-ad1c-40d3-bb78-3127660fcf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:46.788322 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1366/2762] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:46.793464 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17df377-7edb-4328-b57f-b513919e7a75 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:46.860802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:46.891908 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17df377-7edb-4328-b57f-b513919e7a75 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:46.897149 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92feaf-41e2-4d21-b0e0-418d106c78a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:46.897757 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92feaf-41e2-4d21-b0e0-418d106c78a1 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:46.898042 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92feaf-41e2-4d21-b0e0-418d106c78a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:46.898333 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92feaf-41e2-4d21-b0e0-418d106c78a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:46.899032 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92feaf-41e2-4d21-b0e0-418d106c78a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:46.900975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92feaf-41e2-4d21-b0e0-418d106c78a1 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:47.027541 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1397/2763] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:46 2022] POST /identity/v3/projects => generated 385 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:47.033141 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e9ba7-ad1c-40d3-bb78-3127660fcf48 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:47.035531 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-910e9ba7-ad1c-40d3-bb78-3127660fcf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:47.036597 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92feaf-41e2-4d21-b0e0-418d106c78a1 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:47.066424 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e9ba7-ad1c-40d3-bb78-3127660fcf48 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:47.123975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92feaf-41e2-4d21-b0e0-418d106c78a1 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:47.135568 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1991c-6de6-4367-bf39-421e57a9a64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:47.135568 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1991c-6de6-4367-bf39-421e57a9a64b admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:47.135568 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1991c-6de6-4367-bf39-421e57a9a64b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:47.135568 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1991c-6de6-4367-bf39-421e57a9a64b admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:47.136047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1991c-6de6-4367-bf39-421e57a9a64b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:47.138811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb1991c-6de6-4367-bf39-421e57a9a64b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:47.141772 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:47.149204 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83946717-ac58-4b5d-911d-38026f4012fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:47.149946 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83946717-ac58-4b5d-911d-38026f4012fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:47.150443 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83946717-ac58-4b5d-911d-38026f4012fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:47.150790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83946717-ac58-4b5d-911d-38026f4012fb admin admin] PATH_INFO: `/v3/users/b4cc11524fe542c5a5361b3630e6d3a6` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:47.152220 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83946717-ac58-4b5d-911d-38026f4012fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4cc11524fe542c5a5361b3630e6d3a6)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:47.155846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83946717-ac58-4b5d-911d-38026f4012fb admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:47.244527 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1398/2764] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:47 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:47.246707 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1991c-6de6-4367-bf39-421e57a9a64b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:47.287598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb1991c-6de6-4367-bf39-421e57a9a64b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:47.296224 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ea4d6-a7a4-4c63-bf45-55bac32666b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:47.296919 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ea4d6-a7a4-4c63-bf45-55bac32666b7 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:47.301571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ea4d6-a7a4-4c63-bf45-55bac32666b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:47.301571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ea4d6-a7a4-4c63-bf45-55bac32666b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:47.301571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ea4d6-a7a4-4c63-bf45-55bac32666b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:47.301571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ea4d6-a7a4-4c63-bf45-55bac32666b7 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:47.324374 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1399/2765] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:47.328835 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ea4d6-a7a4-4c63-bf45-55bac32666b7 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:47.340127 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ea4d6-a7a4-4c63-bf45-55bac32666b7 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:47.345704 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9d65b-e4d2-455a-bc84-e8dcdcbebae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:47.346385 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9d65b-e4d2-455a-bc84-e8dcdcbebae8 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:47.346731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9d65b-e4d2-455a-bc84-e8dcdcbebae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:47.347107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db9d65b-e4d2-455a-bc84-e8dcdcbebae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:47.347847 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9d65b-e4d2-455a-bc84-e8dcdcbebae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:47.349503 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db9d65b-e4d2-455a-bc84-e8dcdcbebae8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:47.361843 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1400/2766] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:47.362454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9d65b-e4d2-455a-bc84-e8dcdcbebae8 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:47.365552 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2db9d65b-e4d2-455a-bc84-e8dcdcbebae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:47.381089 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-83946717-ac58-4b5d-911d-38026f4012fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4cc11524fe542c5a5361b3630e6d3a6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:47.428556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db9d65b-e4d2-455a-bc84-e8dcdcbebae8 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:47.435815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b041cb8-1257-449c-8105-a3dfe8cf01c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:47.435815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b041cb8-1257-449c-8105-a3dfe8cf01c0 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:47.435815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b041cb8-1257-449c-8105-a3dfe8cf01c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:47.435815 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b041cb8-1257-449c-8105-a3dfe8cf01c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:47.436251 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5b041cb8-1257-449c-8105-a3dfe8cf01c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:47.456065 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-83946717-ac58-4b5d-911d-38026f4012fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4cc11524fe542c5a5361b3630e6d3a6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:47.469469 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-83946717-ac58-4b5d-911d-38026f4012fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4cc11524fe542c5a5361b3630e6d3a6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:47.470200 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-83946717-ac58-4b5d-911d-38026f4012fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4cc11524fe542c5a5361b3630e6d3a6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:47.484574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-83946717-ac58-4b5d-911d-38026f4012fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4cc11524fe542c5a5361b3630e6d3a6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:47.488896 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:47.491578 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1367/2767] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:47 2022] DELETE /identity/v3/users/b4cc11524fe542c5a5361b3630e6d3a6 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:47.494672 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83946717-ac58-4b5d-911d-38026f4012fb admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:47.495162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-83946717-ac58-4b5d-911d-38026f4012fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:47.507565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b041cb8-1257-449c-8105-a3dfe8cf01c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:47.509040 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b041cb8-1257-449c-8105-a3dfe8cf01c0 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:47.518682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1401/2768] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:47.574448 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83946717-ac58-4b5d-911d-38026f4012fb admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:47.581533 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd88db6-2591-4fa1-84c2-2d872a1936bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:47.582385 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd88db6-2591-4fa1-84c2-2d872a1936bd admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:47.586582 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd88db6-2591-4fa1-84c2-2d872a1936bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:47.586582 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd88db6-2591-4fa1-84c2-2d872a1936bd admin admin] PATH_INFO: `/v3/projects/90301db9896e4c2a9cd37841bb4e2d00/users/4cc00fcf1060416899cc7b0cb1165671/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:47.603747 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd88db6-2591-4fa1-84c2-2d872a1936bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=90301db9896e4c2a9cd37841bb4e2d00, user_id=4cc00fcf1060416899cc7b0cb1165671, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:47.608156 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd88db6-2591-4fa1-84c2-2d872a1936bd admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:47.740106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1368/2769] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:47 2022] PUT /identity/v3/projects/90301db9896e4c2a9cd37841bb4e2d00/users/4cc00fcf1060416899cc7b0cb1165671/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:47.750441 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d33fa6-6dea-4fb0-984b-c6de4481130e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:47.750956 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d33fa6-6dea-4fb0-984b-c6de4481130e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:47.751081 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d33fa6-6dea-4fb0-984b-c6de4481130e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:47.779963 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a5d33fa6-6dea-4fb0-984b-c6de4481130e None None] MFA Rules not processed for user `4cc00fcf1060416899cc7b0cb1165671`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:47.781187 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a5d33fa6-6dea-4fb0-984b-c6de4481130e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:47.824738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd88db6-2591-4fa1-84c2-2d872a1936bd admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:47.827181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6bd88db6-2591-4fa1-84c2-2d872a1936bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:47.897949 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1402/2770] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:47.899143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d33fa6-6dea-4fb0-984b-c6de4481130e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:47.961888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:47.966587 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd88db6-2591-4fa1-84c2-2d872a1936bd admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:47.972036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ebb36-12d7-4299-bdf9-9c010d1f302f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:47.972667 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ebb36-12d7-4299-bdf9-9c010d1f302f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:47.973041 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ebb36-12d7-4299-bdf9-9c010d1f302f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:47.973225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ebb36-12d7-4299-bdf9-9c010d1f302f admin admin] PATH_INFO: `/v3/projects/fefb0f83ce4447cc9223c383a256676a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:47.974385 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ebb36-12d7-4299-bdf9-9c010d1f302f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fefb0f83ce4447cc9223c383a256676a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:47.976330 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ebb36-12d7-4299-bdf9-9c010d1f302f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:48.020087 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d33fa6-6dea-4fb0-984b-c6de4481130e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:48.027994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cddf45-627d-4572-92c8-4e9e468a29a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:48.028948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cddf45-627d-4572-92c8-4e9e468a29a8 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:48.029438 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cddf45-627d-4572-92c8-4e9e468a29a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:48.029926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cddf45-627d-4572-92c8-4e9e468a29a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:48.030659 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cddf45-627d-4572-92c8-4e9e468a29a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:48.033073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cddf45-627d-4572-92c8-4e9e468a29a8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:48.140679 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d70ebb36-12d7-4299-bdf9-9c010d1f302f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fefb0f83ce4447cc9223c383a256676a', 'request_id': 'req-d70ebb36-12d7-4299-bdf9-9c010d1f302f'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:48.147968 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d70ebb36-12d7-4299-bdf9-9c010d1f302f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fefb0f83ce4447cc9223c383a256676a', 'request_id': 'req-d70ebb36-12d7-4299-bdf9-9c010d1f302f'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:48.154698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1403/2771] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:47 2022] POST /identity/v3/projects => generated 385 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:48.158530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cddf45-627d-4572-92c8-4e9e468a29a8 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:48.160653 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d70ebb36-12d7-4299-bdf9-9c010d1f302f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fefb0f83ce4447cc9223c383a256676a', 'request_id': 'req-d70ebb36-12d7-4299-bdf9-9c010d1f302f'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:48.247868 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d70ebb36-12d7-4299-bdf9-9c010d1f302f admin admin] The token cache is being invalidate because project fefb0f83ce4447cc9223c383a256676a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:48.248181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d70ebb36-12d7-4299-bdf9-9c010d1f302f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:48.249132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d70ebb36-12d7-4299-bdf9-9c010d1f302f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:48.252358 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1369/2772] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:47 2022] DELETE /identity/v3/projects/fefb0f83ce4447cc9223c383a256676a => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:48.258863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057de557-d5ac-4db5-86a9-7fac6927ff22 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:48.259251 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057de557-d5ac-4db5-86a9-7fac6927ff22 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:48.259595 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057de557-d5ac-4db5-86a9-7fac6927ff22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:48.265081 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:48.269386 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cddf45-627d-4572-92c8-4e9e468a29a8 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:48.274536 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3fd65-0234-4656-84bf-98580a0a0fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:48.275191 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3fd65-0234-4656-84bf-98580a0a0fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:48.275472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3fd65-0234-4656-84bf-98580a0a0fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:48.275748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3fd65-0234-4656-84bf-98580a0a0fd8 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:48.276382 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3fd65-0234-4656-84bf-98580a0a0fd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:48.278340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3fd65-0234-4656-84bf-98580a0a0fd8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:48.283558 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-057de557-d5ac-4db5-86a9-7fac6927ff22 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:48.284774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-057de557-d5ac-4db5-86a9-7fac6927ff22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:48.295351 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1370/2773] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:48.303066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5ab9d-2bf2-46eb-bb39-58a9a2908198 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:48.303348 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5ab9d-2bf2-46eb-bb39-58a9a2908198 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:48.303621 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5ab9d-2bf2-46eb-bb39-58a9a2908198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:48.331226 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-08f5ab9d-2bf2-46eb-bb39-58a9a2908198 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:48.332512 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-08f5ab9d-2bf2-46eb-bb39-58a9a2908198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:48.349829 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1371/2774] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:48.354908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5ab9d-2bf2-46eb-bb39-58a9a2908198 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:48.372387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5ab9d-2bf2-46eb-bb39-58a9a2908198 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:48.377610 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad513d-096c-4a18-8216-2a85c026a8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:48.383466 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad513d-096c-4a18-8216-2a85c026a8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:48.383466 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad513d-096c-4a18-8216-2a85c026a8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:48.383466 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad513d-096c-4a18-8216-2a85c026a8e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:48.383887 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1404/2775] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:48 2022] POST /identity/v3/users => generated 299 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:48.383992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad513d-096c-4a18-8216-2a85c026a8e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:08:48.383992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad513d-096c-4a18-8216-2a85c026a8e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:48.383992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad513d-096c-4a18-8216-2a85c026a8e4 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:48.390618 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3fd65-0234-4656-84bf-98580a0a0fd8 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:48.392997 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-30b3fd65-0234-4656-84bf-98580a0a0fd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:48.395179 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1372/2776] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:08:48.399766 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad513d-096c-4a18-8216-2a85c026a8e4 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:48.413253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3fd65-0234-4656-84bf-98580a0a0fd8 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:48.413438 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad513d-096c-4a18-8216-2a85c026a8e4 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:48.420225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77074c67-6e5f-4085-a39c-7564dec035c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:48.420225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77074c67-6e5f-4085-a39c-7564dec035c6 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:48.420225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77074c67-6e5f-4085-a39c-7564dec035c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:48.420225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77074c67-6e5f-4085-a39c-7564dec035c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:48.421354 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77074c67-6e5f-4085-a39c-7564dec035c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:48.423165 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77074c67-6e5f-4085-a39c-7564dec035c6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:48.424004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa4acb-c549-4e61-80a9-f7832b36466c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:48.424631 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa4acb-c549-4e61-80a9-f7832b36466c admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:48.424907 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa4acb-c549-4e61-80a9-f7832b36466c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:48.425178 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa4acb-c549-4e61-80a9-f7832b36466c admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:48.426248 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa4acb-c549-4e61-80a9-f7832b36466c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:48.428226 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aa4acb-c549-4e61-80a9-f7832b36466c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:48.437121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1405/2777] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:48.442947 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77074c67-6e5f-4085-a39c-7564dec035c6 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:48.457490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77074c67-6e5f-4085-a39c-7564dec035c6 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:48.463225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e267fb-c10d-4e72-9702-e30cfb174538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:48.463994 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e267fb-c10d-4e72-9702-e30cfb174538 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:48.465494 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e267fb-c10d-4e72-9702-e30cfb174538 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:48.465843 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e267fb-c10d-4e72-9702-e30cfb174538 admin admin] PATH_INFO: `/v3/projects/ba2384b621474b73ae6bd8ad9e3054e8/users/8b8fb3ca05db4960821c9f060d4cf032/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:48.484801 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e267fb-c10d-4e72-9702-e30cfb174538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2384b621474b73ae6bd8ad9e3054e8, user_id=8b8fb3ca05db4960821c9f060d4cf032, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:48.487444 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e267fb-c10d-4e72-9702-e30cfb174538 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:48.512929 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1373/2778] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:48 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:48.517052 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa4acb-c549-4e61-80a9-f7832b36466c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:48.557271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1406/2779] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:48 2022] PUT /identity/v3/projects/ba2384b621474b73ae6bd8ad9e3054e8/users/8b8fb3ca05db4960821c9f060d4cf032/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:48.567271 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e267fb-c10d-4e72-9702-e30cfb174538 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:48.601490 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa4acb-c549-4e61-80a9-f7832b36466c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:48.648870 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e267fb-c10d-4e72-9702-e30cfb174538 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:48.656100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757a8d7-1a60-4e73-932d-f07a46978b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:48.656822 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757a8d7-1a60-4e73-932d-f07a46978b60 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:48.657108 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757a8d7-1a60-4e73-932d-f07a46978b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:48.657380 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757a8d7-1a60-4e73-932d-f07a46978b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:48.658025 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757a8d7-1a60-4e73-932d-f07a46978b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:48.660274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757a8d7-1a60-4e73-932d-f07a46978b60 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:48.669997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:48.676816 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1407/2780] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:48.677999 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757a8d7-1a60-4e73-932d-f07a46978b60 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:48.684499 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe03f6-7d3f-4409-b8b2-1ca38098ae3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:48.685125 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe03f6-7d3f-4409-b8b2-1ca38098ae3e admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:48.685407 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe03f6-7d3f-4409-b8b2-1ca38098ae3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:48.685688 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe03f6-7d3f-4409-b8b2-1ca38098ae3e admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:48.686332 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe03f6-7d3f-4409-b8b2-1ca38098ae3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:48.688304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe03f6-7d3f-4409-b8b2-1ca38098ae3e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:48.688877 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757a8d7-1a60-4e73-932d-f07a46978b60 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:48.693677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2368e677-bc42-4b91-9005-7a0201a08c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:48.694343 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368e677-bc42-4b91-9005-7a0201a08c5f admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:48.694636 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368e677-bc42-4b91-9005-7a0201a08c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:48.694926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2368e677-bc42-4b91-9005-7a0201a08c5f admin admin] PATH_INFO: `/v3/projects/ba2384b621474b73ae6bd8ad9e3054e8/users/8b8fb3ca05db4960821c9f060d4cf032/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:48.696956 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368e677-bc42-4b91-9005-7a0201a08c5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2384b621474b73ae6bd8ad9e3054e8, user_id=8b8fb3ca05db4960821c9f060d4cf032, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:48.699685 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2368e677-bc42-4b91-9005-7a0201a08c5f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:48.710826 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1408/2781] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:48 2022] PUT /identity/v3/projects/ba2384b621474b73ae6bd8ad9e3054e8/users/8b8fb3ca05db4960821c9f060d4cf032/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:08:48.721898 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b8d9a-cea8-4e1e-8efb-70877f478361 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:48.722184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b8d9a-cea8-4e1e-8efb-70877f478361 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:48.722460 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b8d9a-cea8-4e1e-8efb-70877f478361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:48.751020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f64b8d9a-cea8-4e1e-8efb-70877f478361 None None] MFA Rules not processed for user `8b8fb3ca05db4960821c9f060d4cf032`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:48.752238 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f64b8d9a-cea8-4e1e-8efb-70877f478361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:48.814099 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1374/2782] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:48 2022] POST /identity/v3/users => generated 285 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:08:48.820632 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe03f6-7d3f-4409-b8b2-1ca38098ae3e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:48.838225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1409/2783] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:48.843311 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe03f6-7d3f-4409-b8b2-1ca38098ae3e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:48.854867 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b8d9a-cea8-4e1e-8efb-70877f478361 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:48.857465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f64b8d9a-cea8-4e1e-8efb-70877f478361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:48.885682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:48.941944 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b8d9a-cea8-4e1e-8efb-70877f478361 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:48.944715 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4fee0-bf93-44a6-93ad-ac2b82221253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:48.945461 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4fee0-bf93-44a6-93ad-ac2b82221253 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:48.945739 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4fee0-bf93-44a6-93ad-ac2b82221253 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:48.946082 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4fee0-bf93-44a6-93ad-ac2b82221253 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:48.946646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4fee0-bf93-44a6-93ad-ac2b82221253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:48.947705 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d73259-a545-489c-890d-e341de85350d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:48.949229 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4fee0-bf93-44a6-93ad-ac2b82221253 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:48.949392 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d73259-a545-489c-890d-e341de85350d service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:48.949720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d73259-a545-489c-890d-e341de85350d service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:48.950193 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d73259-a545-489c-890d-e341de85350d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:48.965072 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1375/2784] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:48.966580 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4fee0-bf93-44a6-93ad-ac2b82221253 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:48.974772 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d73259-a545-489c-890d-e341de85350d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:48.977353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d73259-a545-489c-890d-e341de85350d service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:48.980492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4fee0-bf93-44a6-93ad-ac2b82221253 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:48.986312 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38383699-b7b3-4814-9741-73eb08368b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:48.987155 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38383699-b7b3-4814-9741-73eb08368b92 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:48.987692 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38383699-b7b3-4814-9741-73eb08368b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:48.988087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38383699-b7b3-4814-9741-73eb08368b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:48.988805 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38383699-b7b3-4814-9741-73eb08368b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:48.990524 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38383699-b7b3-4814-9741-73eb08368b92 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:48.993895 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1410/2785] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:08:49.004196 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1376/2786] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:49.008340 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d73259-a545-489c-890d-e341de85350d service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:49.020727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d73259-a545-489c-890d-e341de85350d service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:49.028453 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd6cee-06ed-4874-a71a-71a99f24ec4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:49.029120 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd6cee-06ed-4874-a71a-71a99f24ec4e admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:49.029402 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd6cee-06ed-4874-a71a-71a99f24ec4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:49.029679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd6cee-06ed-4874-a71a-71a99f24ec4e admin admin] PATH_INFO: `/v3/projects/30b7603d3bf84c8fbfa5b77ff92b9347/users/ec2583029bc94c208ffa8502e38e4f3a/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:49.051759 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd6cee-06ed-4874-a71a-71a99f24ec4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=30b7603d3bf84c8fbfa5b77ff92b9347, user_id=ec2583029bc94c208ffa8502e38e4f3a, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:49.054457 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd6cee-06ed-4874-a71a-71a99f24ec4e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:49.141755 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1411/2787] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:49 2022] PUT /identity/v3/projects/30b7603d3bf84c8fbfa5b77ff92b9347/users/ec2583029bc94c208ffa8502e38e4f3a/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:49.160394 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1806375-1546-49da-8c0f-4bb14425a062 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:49.160846 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1806375-1546-49da-8c0f-4bb14425a062 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:49.160929 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1806375-1546-49da-8c0f-4bb14425a062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:49.192169 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f1806375-1546-49da-8c0f-4bb14425a062 None None] MFA Rules not processed for user `ec2583029bc94c208ffa8502e38e4f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:49.193387 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f1806375-1546-49da-8c0f-4bb14425a062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:49.251432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd6cee-06ed-4874-a71a-71a99f24ec4e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:49.253668 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b2cd6cee-06ed-4874-a71a-71a99f24ec4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:49.284719 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1377/2788] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:49.302356 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1806375-1546-49da-8c0f-4bb14425a062 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:49.304061 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f1806375-1546-49da-8c0f-4bb14425a062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:49.335627 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd6cee-06ed-4874-a71a-71a99f24ec4e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:49.339624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971cfd0-bb9b-440f-931f-c949ba2dee71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:49.340358 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971cfd0-bb9b-440f-931f-c949ba2dee71 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:49.340731 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971cfd0-bb9b-440f-931f-c949ba2dee71 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:49.340931 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971cfd0-bb9b-440f-931f-c949ba2dee71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:49.342827 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2971cfd0-bb9b-440f-931f-c949ba2dee71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:49.377571 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1806375-1546-49da-8c0f-4bb14425a062 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:49.382197 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b900cec-3786-4f48-a539-2b76c384a5e2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01b060c70e1b440186eeac37d9c34567', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:49.383053 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b900cec-3786-4f48-a539-2b76c384a5e2 service glance] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:49.383964 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b900cec-3786-4f48-a539-2b76c384a5e2 service glance] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:49.384538 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b900cec-3786-4f48-a539-2b76c384a5e2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:49.394477 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:49.399565 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b900cec-3786-4f48-a539-2b76c384a5e2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:49.403943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b900cec-3786-4f48-a539-2b76c384a5e2 service glance] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:49.416413 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1378/2789] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:49.440718 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971cfd0-bb9b-440f-931f-c949ba2dee71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:49.442283 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971cfd0-bb9b-440f-931f-c949ba2dee71 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:49.468759 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1412/2790] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:08:50.194771 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b900cec-3786-4f48-a539-2b76c384a5e2 service glance] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:50.204608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b900cec-3786-4f48-a539-2b76c384a5e2 service glance] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:50.208673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0f3a1-3cea-4dcc-a722-fe1c636d42d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:50.208984 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0f3a1-3cea-4dcc-a722-fe1c636d42d6 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:50.209097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0f3a1-3cea-4dcc-a722-fe1c636d42d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:50.209367 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0f3a1-3cea-4dcc-a722-fe1c636d42d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:50.211019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5fd0f3a1-3cea-4dcc-a722-fe1c636d42d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:50.269958 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0f3a1-3cea-4dcc-a722-fe1c636d42d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:50.271341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0f3a1-3cea-4dcc-a722-fe1c636d42d6 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:50.280901 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1379/2791] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:54.519227 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971cfd0-bb9b-440f-931f-c949ba2dee71 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:54.520681 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2971cfd0-bb9b-440f-931f-c949ba2dee71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:54.575603 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:54.597208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971cfd0-bb9b-440f-931f-c949ba2dee71 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:54.603307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba6f08-b116-4109-b785-15432120eaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:54.606039 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba6f08-b116-4109-b785-15432120eaf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:54.606039 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba6f08-b116-4109-b785-15432120eaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:54.606039 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba6f08-b116-4109-b785-15432120eaf3 admin admin] PATH_INFO: `/v3/users/7e6e5ac0689b47aea4e79192454fd04a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:54.607569 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba6f08-b116-4109-b785-15432120eaf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e6e5ac0689b47aea4e79192454fd04a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:54.610970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba6f08-b116-4109-b785-15432120eaf3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:54.830780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-37ba6f08-b116-4109-b785-15432120eaf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e6e5ac0689b47aea4e79192454fd04a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:54.916149 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-37ba6f08-b116-4109-b785-15432120eaf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e6e5ac0689b47aea4e79192454fd04a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:54.916934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-37ba6f08-b116-4109-b785-15432120eaf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e6e5ac0689b47aea4e79192454fd04a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:54.925878 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-37ba6f08-b116-4109-b785-15432120eaf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e6e5ac0689b47aea4e79192454fd04a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:54.939403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-37ba6f08-b116-4109-b785-15432120eaf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e6e5ac0689b47aea4e79192454fd04a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:54.945529 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1413/2792] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:54 2022] DELETE /identity/v3/users/7e6e5ac0689b47aea4e79192454fd04a => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:54.951042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a0e52-86ed-43f5-9d8b-f5d7e7aaad4e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:54.951549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a0e52-86ed-43f5-9d8b-f5d7e7aaad4e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:54.951665 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a0e52-86ed-43f5-9d8b-f5d7e7aaad4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:54.971826 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-833a0e52-86ed-43f5-9d8b-f5d7e7aaad4e None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:54.972704 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-833a0e52-86ed-43f5-9d8b-f5d7e7aaad4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:55.035225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1380/2793] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:55.045369 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba6f08-b116-4109-b785-15432120eaf3 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:55.046873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-37ba6f08-b116-4109-b785-15432120eaf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:55.112170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba6f08-b116-4109-b785-15432120eaf3 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:55.116235 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f031d-8d31-48a9-89d7-a306293cb6b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:55.116841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f031d-8d31-48a9-89d7-a306293cb6b2 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:55.117120 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f031d-8d31-48a9-89d7-a306293cb6b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:55.117435 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f031d-8d31-48a9-89d7-a306293cb6b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:55.127413 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f031d-8d31-48a9-89d7-a306293cb6b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:55.129047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f031d-8d31-48a9-89d7-a306293cb6b2 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:55.140091 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1414/2794] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:55.577279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a0e52-86ed-43f5-9d8b-f5d7e7aaad4e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:55.578696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-833a0e52-86ed-43f5-9d8b-f5d7e7aaad4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:55.592709 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a0e52-86ed-43f5-9d8b-f5d7e7aaad4e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:55.597516 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce47c8c-93c5-40d5-bad8-216ec52e393c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:55.598152 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce47c8c-93c5-40d5-bad8-216ec52e393c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:55.598432 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce47c8c-93c5-40d5-bad8-216ec52e393c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:55.598708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce47c8c-93c5-40d5-bad8-216ec52e393c admin admin] PATH_INFO: `/v3/projects/c1aa7132baef46debf5f9f5e6f2885ef` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:55.599857 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce47c8c-93c5-40d5-bad8-216ec52e393c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1aa7132baef46debf5f9f5e6f2885ef)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:55.601877 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce47c8c-93c5-40d5-bad8-216ec52e393c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:55.672197 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f031d-8d31-48a9-89d7-a306293cb6b2 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:55.673720 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d91f031d-8d31-48a9-89d7-a306293cb6b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:55.716339 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5ce47c8c-93c5-40d5-bad8-216ec52e393c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1aa7132baef46debf5f9f5e6f2885ef', 'request_id': 'req-5ce47c8c-93c5-40d5-bad8-216ec52e393c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:55.725273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5ce47c8c-93c5-40d5-bad8-216ec52e393c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1aa7132baef46debf5f9f5e6f2885ef', 'request_id': 'req-5ce47c8c-93c5-40d5-bad8-216ec52e393c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:55.733223 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5ce47c8c-93c5-40d5-bad8-216ec52e393c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1aa7132baef46debf5f9f5e6f2885ef', 'request_id': 'req-5ce47c8c-93c5-40d5-bad8-216ec52e393c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:55.748105 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f031d-8d31-48a9-89d7-a306293cb6b2 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:55.774521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:55.802220 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5ce47c8c-93c5-40d5-bad8-216ec52e393c admin admin] The token cache is being invalidate because project c1aa7132baef46debf5f9f5e6f2885ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:55.802506 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5ce47c8c-93c5-40d5-bad8-216ec52e393c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:55.805539 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5ce47c8c-93c5-40d5-bad8-216ec52e393c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:55.805539 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1381/2795] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:55 2022] DELETE /identity/v3/projects/c1aa7132baef46debf5f9f5e6f2885ef => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:55.806750 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f3a05-f071-4222-8a87-486724463e66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:55.807404 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f3a05-f071-4222-8a87-486724463e66 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:55.807677 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f3a05-f071-4222-8a87-486724463e66 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:55.807944 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f3a05-f071-4222-8a87-486724463e66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:55.808924 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce47c8c-93c5-40d5-bad8-216ec52e393c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:55.810776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f11f3a05-f071-4222-8a87-486724463e66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:55.810965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5ce47c8c-93c5-40d5-bad8-216ec52e393c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:55.834418 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:55.907521 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f3a05-f071-4222-8a87-486724463e66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:55.909053 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f3a05-f071-4222-8a87-486724463e66 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:55.920784 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce47c8c-93c5-40d5-bad8-216ec52e393c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:55.925308 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c87250-19a6-436d-9ddd-2c7dd88cbe47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:55.925981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1415/2796] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:55.926108 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c87250-19a6-436d-9ddd-2c7dd88cbe47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:55.926362 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c87250-19a6-436d-9ddd-2c7dd88cbe47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:55.926640 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c87250-19a6-436d-9ddd-2c7dd88cbe47 admin admin] PATH_INFO: `/v3/users/124fa9afc19f41ddb6d1989b19d550f9` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:55.927913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c87250-19a6-436d-9ddd-2c7dd88cbe47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=124fa9afc19f41ddb6d1989b19d550f9)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:55.930106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c87250-19a6-436d-9ddd-2c7dd88cbe47 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:56.142583 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-36c87250-19a6-436d-9ddd-2c7dd88cbe47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '124fa9afc19f41ddb6d1989b19d550f9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:56.221316 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-36c87250-19a6-436d-9ddd-2c7dd88cbe47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '124fa9afc19f41ddb6d1989b19d550f9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:56.237133 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-36c87250-19a6-436d-9ddd-2c7dd88cbe47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '124fa9afc19f41ddb6d1989b19d550f9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:56.238276 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-36c87250-19a6-436d-9ddd-2c7dd88cbe47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '124fa9afc19f41ddb6d1989b19d550f9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:56.245157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-36c87250-19a6-436d-9ddd-2c7dd88cbe47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '124fa9afc19f41ddb6d1989b19d550f9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:56.251366 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1382/2797] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:55 2022] DELETE /identity/v3/users/124fa9afc19f41ddb6d1989b19d550f9 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:56.720548 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f3a05-f071-4222-8a87-486724463e66 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:56.734259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f3a05-f071-4222-8a87-486724463e66 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:56.801717 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16321d7e-a7a5-401b-84f6-14ec4493777f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:56.802348 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16321d7e-a7a5-401b-84f6-14ec4493777f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:56.802628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16321d7e-a7a5-401b-84f6-14ec4493777f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:56.802913 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16321d7e-a7a5-401b-84f6-14ec4493777f admin admin] PATH_INFO: `/v3/projects/6ca780589ff14de98c17aa08d581c2e7` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:56.804038 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16321d7e-a7a5-401b-84f6-14ec4493777f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ca780589ff14de98c17aa08d581c2e7)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:56.805978 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16321d7e-a7a5-401b-84f6-14ec4493777f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:56.991068 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-16321d7e-a7a5-401b-84f6-14ec4493777f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ca780589ff14de98c17aa08d581c2e7', 'request_id': 'req-16321d7e-a7a5-401b-84f6-14ec4493777f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:57.064898 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-16321d7e-a7a5-401b-84f6-14ec4493777f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ca780589ff14de98c17aa08d581c2e7', 'request_id': 'req-16321d7e-a7a5-401b-84f6-14ec4493777f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:57.072933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-16321d7e-a7a5-401b-84f6-14ec4493777f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ca780589ff14de98c17aa08d581c2e7', 'request_id': 'req-16321d7e-a7a5-401b-84f6-14ec4493777f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:57.086473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-16321d7e-a7a5-401b-84f6-14ec4493777f admin admin] The token cache is being invalidate because project 6ca780589ff14de98c17aa08d581c2e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:57.086768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-16321d7e-a7a5-401b-84f6-14ec4493777f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:57.087458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-16321d7e-a7a5-401b-84f6-14ec4493777f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:57.092201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1416/2798] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:56 2022] DELETE /identity/v3/projects/6ca780589ff14de98c17aa08d581c2e7 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:57.096762 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c87250-19a6-436d-9ddd-2c7dd88cbe47 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:57.099578 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-36c87250-19a6-436d-9ddd-2c7dd88cbe47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:57.133109 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c87250-19a6-436d-9ddd-2c7dd88cbe47 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:57.200500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78ecec-89ad-4231-a23b-fc71d5837bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:57.201139 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78ecec-89ad-4231-a23b-fc71d5837bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:57.201404 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78ecec-89ad-4231-a23b-fc71d5837bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:57.201673 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78ecec-89ad-4231-a23b-fc71d5837bd0 admin admin] PATH_INFO: `/v3/users/badb1f2fb7ca4c6ea5c8be29a2b347c5` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:57.202814 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78ecec-89ad-4231-a23b-fc71d5837bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=badb1f2fb7ca4c6ea5c8be29a2b347c5)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:57.204800 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78ecec-89ad-4231-a23b-fc71d5837bd0 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:57.476320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:57.489556 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7a78ecec-89ad-4231-a23b-fc71d5837bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'badb1f2fb7ca4c6ea5c8be29a2b347c5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:57.541073 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7a78ecec-89ad-4231-a23b-fc71d5837bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'badb1f2fb7ca4c6ea5c8be29a2b347c5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:57.564063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7a78ecec-89ad-4231-a23b-fc71d5837bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'badb1f2fb7ca4c6ea5c8be29a2b347c5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:57.564063 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7a78ecec-89ad-4231-a23b-fc71d5837bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'badb1f2fb7ca4c6ea5c8be29a2b347c5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:57.568923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-7a78ecec-89ad-4231-a23b-fc71d5837bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'badb1f2fb7ca4c6ea5c8be29a2b347c5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:57.580609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1383/2799] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:57 2022] DELETE /identity/v3/users/badb1f2fb7ca4c6ea5c8be29a2b347c5 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:58.063184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16321d7e-a7a5-401b-84f6-14ec4493777f admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:58.064686 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-16321d7e-a7a5-401b-84f6-14ec4493777f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:58.078225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16321d7e-a7a5-401b-84f6-14ec4493777f admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:58.099667 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:58.156264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48763d92-3bb3-43ed-b7a0-62dde836d391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:58.157137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48763d92-3bb3-43ed-b7a0-62dde836d391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:58.157619 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48763d92-3bb3-43ed-b7a0-62dde836d391 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:58.158089 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48763d92-3bb3-43ed-b7a0-62dde836d391 admin admin] PATH_INFO: `/v3/projects/4bc79e09daee475fa44b1d20cf86896d` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:58.160220 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48763d92-3bb3-43ed-b7a0-62dde836d391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bc79e09daee475fa44b1d20cf86896d)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:58.162143 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48763d92-3bb3-43ed-b7a0-62dde836d391 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:58.344956 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-48763d92-3bb3-43ed-b7a0-62dde836d391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bc79e09daee475fa44b1d20cf86896d', 'request_id': 'req-48763d92-3bb3-43ed-b7a0-62dde836d391'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:58.415608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-48763d92-3bb3-43ed-b7a0-62dde836d391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bc79e09daee475fa44b1d20cf86896d', 'request_id': 'req-48763d92-3bb3-43ed-b7a0-62dde836d391'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:58.428446 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-48763d92-3bb3-43ed-b7a0-62dde836d391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bc79e09daee475fa44b1d20cf86896d', 'request_id': 'req-48763d92-3bb3-43ed-b7a0-62dde836d391'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:58.441391 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-48763d92-3bb3-43ed-b7a0-62dde836d391 admin admin] The token cache is being invalidate because project 4bc79e09daee475fa44b1d20cf86896d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:58.441682 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-48763d92-3bb3-43ed-b7a0-62dde836d391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:58.442389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-48763d92-3bb3-43ed-b7a0-62dde836d391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:58.444129 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1417/2800] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:58 2022] DELETE /identity/v3/projects/4bc79e09daee475fa44b1d20cf86896d => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:00.453988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78ecec-89ad-4231-a23b-fc71d5837bd0 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:00.456007 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7a78ecec-89ad-4231-a23b-fc71d5837bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:00.469857 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78ecec-89ad-4231-a23b-fc71d5837bd0 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:00.534727 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb4fb7-0cb0-46f3-b956-3dfd1e8f03e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:00.535397 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb4fb7-0cb0-46f3-b956-3dfd1e8f03e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:00.535674 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb4fb7-0cb0-46f3-b956-3dfd1e8f03e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:00.535946 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb4fb7-0cb0-46f3-b956-3dfd1e8f03e4 admin admin] PATH_INFO: `/v3/users/3427af5d220c4c3c9c5a7bdb913f54f0` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:00.537111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb4fb7-0cb0-46f3-b956-3dfd1e8f03e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3427af5d220c4c3c9c5a7bdb913f54f0)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:00.539087 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb4fb7-0cb0-46f3-b956-3dfd1e8f03e4 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:00.726864 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-59bb4fb7-0cb0-46f3-b956-3dfd1e8f03e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3427af5d220c4c3c9c5a7bdb913f54f0'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:00.815753 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-59bb4fb7-0cb0-46f3-b956-3dfd1e8f03e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3427af5d220c4c3c9c5a7bdb913f54f0'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:00.830036 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-59bb4fb7-0cb0-46f3-b956-3dfd1e8f03e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3427af5d220c4c3c9c5a7bdb913f54f0'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:00.830957 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-59bb4fb7-0cb0-46f3-b956-3dfd1e8f03e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3427af5d220c4c3c9c5a7bdb913f54f0'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:00.838189 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-59bb4fb7-0cb0-46f3-b956-3dfd1e8f03e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3427af5d220c4c3c9c5a7bdb913f54f0'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:00.844613 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1384/2801] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:09:00 2022] DELETE /identity/v3/users/3427af5d220c4c3c9c5a7bdb913f54f0 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:00.849144 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f9283-bf83-4b81-8a5e-51dbcb8abac2 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:00.849630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f9283-bf83-4b81-8a5e-51dbcb8abac2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:00.852305 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f9283-bf83-4b81-8a5e-51dbcb8abac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:00.870844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c13f9283-bf83-4b81-8a5e-51dbcb8abac2 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:00.873418 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c13f9283-bf83-4b81-8a5e-51dbcb8abac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:00.944196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1418/2802] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:00.954519 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb4fb7-0cb0-46f3-b956-3dfd1e8f03e4 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:00.956122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-59bb4fb7-0cb0-46f3-b956-3dfd1e8f03e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:00.994321 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:01.020646 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb4fb7-0cb0-46f3-b956-3dfd1e8f03e4 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:01.024574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d690674-4b21-45c9-93c8-7cd50b9f4222 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:01.025195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d690674-4b21-45c9-93c8-7cd50b9f4222 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:01.025549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d690674-4b21-45c9-93c8-7cd50b9f4222 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:01.025773 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d690674-4b21-45c9-93c8-7cd50b9f4222 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:01.036732 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d690674-4b21-45c9-93c8-7cd50b9f4222 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:01.038231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d690674-4b21-45c9-93c8-7cd50b9f4222 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:01.049610 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1385/2803] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:01.508322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f9283-bf83-4b81-8a5e-51dbcb8abac2 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:01.509785 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c13f9283-bf83-4b81-8a5e-51dbcb8abac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:01.524147 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f9283-bf83-4b81-8a5e-51dbcb8abac2 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:01.534835 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd48b1-0e2a-42da-8233-54257c5a9d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:01.534835 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd48b1-0e2a-42da-8233-54257c5a9d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:01.534835 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd48b1-0e2a-42da-8233-54257c5a9d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:01.534835 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd48b1-0e2a-42da-8233-54257c5a9d05 admin admin] PATH_INFO: `/v3/projects/69ae1b2f4e9341f69d5db6b0a4425c25` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:01.535667 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd48b1-0e2a-42da-8233-54257c5a9d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69ae1b2f4e9341f69d5db6b0a4425c25)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:01.535667 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd48b1-0e2a-42da-8233-54257c5a9d05 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:01.643476 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:01.652515 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-62bd48b1-0e2a-42da-8233-54257c5a9d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69ae1b2f4e9341f69d5db6b0a4425c25', 'request_id': 'req-62bd48b1-0e2a-42da-8233-54257c5a9d05'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:01.724263 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-62bd48b1-0e2a-42da-8233-54257c5a9d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69ae1b2f4e9341f69d5db6b0a4425c25', 'request_id': 'req-62bd48b1-0e2a-42da-8233-54257c5a9d05'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:01.731190 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-62bd48b1-0e2a-42da-8233-54257c5a9d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69ae1b2f4e9341f69d5db6b0a4425c25', 'request_id': 'req-62bd48b1-0e2a-42da-8233-54257c5a9d05'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:01.742024 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-62bd48b1-0e2a-42da-8233-54257c5a9d05 admin admin] The token cache is being invalidate because project 69ae1b2f4e9341f69d5db6b0a4425c25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:09:01.742308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-62bd48b1-0e2a-42da-8233-54257c5a9d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:01.743047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-62bd48b1-0e2a-42da-8233-54257c5a9d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:01.744533 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1419/2804] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:09:01 2022] DELETE /identity/v3/projects/69ae1b2f4e9341f69d5db6b0a4425c25 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:01.748861 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d690674-4b21-45c9-93c8-7cd50b9f4222 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:01.758801 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d690674-4b21-45c9-93c8-7cd50b9f4222 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:01.820613 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115af87f-acc9-4167-97a2-b3888a1ebaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:01.821234 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115af87f-acc9-4167-97a2-b3888a1ebaf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:01.821688 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115af87f-acc9-4167-97a2-b3888a1ebaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:01.822012 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115af87f-acc9-4167-97a2-b3888a1ebaf9 admin admin] PATH_INFO: `/v3/users/4cc00fcf1060416899cc7b0cb1165671` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:01.823386 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115af87f-acc9-4167-97a2-b3888a1ebaf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc00fcf1060416899cc7b0cb1165671)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:01.825851 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115af87f-acc9-4167-97a2-b3888a1ebaf9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:02.039726 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-115af87f-acc9-4167-97a2-b3888a1ebaf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc00fcf1060416899cc7b0cb1165671'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:02.101904 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-115af87f-acc9-4167-97a2-b3888a1ebaf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc00fcf1060416899cc7b0cb1165671'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:02.119520 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-115af87f-acc9-4167-97a2-b3888a1ebaf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc00fcf1060416899cc7b0cb1165671'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:02.120618 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-115af87f-acc9-4167-97a2-b3888a1ebaf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc00fcf1060416899cc7b0cb1165671'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:02.134921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-115af87f-acc9-4167-97a2-b3888a1ebaf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc00fcf1060416899cc7b0cb1165671'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:02.143403 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1386/2805] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:09:01 2022] DELETE /identity/v3/users/4cc00fcf1060416899cc7b0cb1165671 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:02.620584 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd48b1-0e2a-42da-8233-54257c5a9d05 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:02.622988 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-62bd48b1-0e2a-42da-8233-54257c5a9d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:02.624556 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115af87f-acc9-4167-97a2-b3888a1ebaf9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:02.627019 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-115af87f-acc9-4167-97a2-b3888a1ebaf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:02.641837 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd48b1-0e2a-42da-8233-54257c5a9d05 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:02.722231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115af87f-acc9-4167-97a2-b3888a1ebaf9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:02.727153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae34021-fc0a-46a4-b71f-3cb16b6aaacb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:02.728267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae34021-fc0a-46a4-b71f-3cb16b6aaacb service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:02.728403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39243abb-20ea-4ec8-b487-2f20f736b9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:02.729126 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39243abb-20ea-4ec8-b487-2f20f736b9a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:02.729260 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae34021-fc0a-46a4-b71f-3cb16b6aaacb service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:02.729428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39243abb-20ea-4ec8-b487-2f20f736b9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:02.729554 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae34021-fc0a-46a4-b71f-3cb16b6aaacb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:02.729714 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39243abb-20ea-4ec8-b487-2f20f736b9a0 admin admin] PATH_INFO: `/v3/projects/90301db9896e4c2a9cd37841bb4e2d00` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:02.731523 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6ae34021-fc0a-46a4-b71f-3cb16b6aaacb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:02.732281 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39243abb-20ea-4ec8-b487-2f20f736b9a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90301db9896e4c2a9cd37841bb4e2d00)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:02.734185 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39243abb-20ea-4ec8-b487-2f20f736b9a0 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:02.752644 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae34021-fc0a-46a4-b71f-3cb16b6aaacb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:02.754528 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae34021-fc0a-46a4-b71f-3cb16b6aaacb service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:02.758827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:02.764948 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1387/2806] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:02.843173 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae34021-fc0a-46a4-b71f-3cb16b6aaacb service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:02.844943 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6ae34021-fc0a-46a4-b71f-3cb16b6aaacb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:02.857429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae34021-fc0a-46a4-b71f-3cb16b6aaacb service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:02.861262 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0ee96-af83-4a81-aa37-e01439d6a334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:02.869725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0ee96-af83-4a81-aa37-e01439d6a334 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:02.869725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0ee96-af83-4a81-aa37-e01439d6a334 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:02.869725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0ee96-af83-4a81-aa37-e01439d6a334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:02.869725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f1b0ee96-af83-4a81-aa37-e01439d6a334 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:02.918100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-39243abb-20ea-4ec8-b487-2f20f736b9a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90301db9896e4c2a9cd37841bb4e2d00', 'request_id': 'req-39243abb-20ea-4ec8-b487-2f20f736b9a0'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:02.958157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0ee96-af83-4a81-aa37-e01439d6a334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:02.959696 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0ee96-af83-4a81-aa37-e01439d6a334 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:02.971964 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1388/2807] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:09:02.978463 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-39243abb-20ea-4ec8-b487-2f20f736b9a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90301db9896e4c2a9cd37841bb4e2d00', 'request_id': 'req-39243abb-20ea-4ec8-b487-2f20f736b9a0'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:02.986180 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-39243abb-20ea-4ec8-b487-2f20f736b9a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90301db9896e4c2a9cd37841bb4e2d00', 'request_id': 'req-39243abb-20ea-4ec8-b487-2f20f736b9a0'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:02.997178 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-39243abb-20ea-4ec8-b487-2f20f736b9a0 admin admin] The token cache is being invalidate because project 90301db9896e4c2a9cd37841bb4e2d00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:09:02.997491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-39243abb-20ea-4ec8-b487-2f20f736b9a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:02.998135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-39243abb-20ea-4ec8-b487-2f20f736b9a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:03.000060 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1420/2808] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:09:02 2022] DELETE /identity/v3/projects/90301db9896e4c2a9cd37841bb4e2d00 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:03.003857 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0ee96-af83-4a81-aa37-e01439d6a334 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:03.005370 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f1b0ee96-af83-4a81-aa37-e01439d6a334 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:03.019947 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0ee96-af83-4a81-aa37-e01439d6a334 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:03.091634 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de67bce-494d-4419-a557-6f21d7534fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:03.092149 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de67bce-494d-4419-a557-6f21d7534fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:03.092461 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de67bce-494d-4419-a557-6f21d7534fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:03.092737 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de67bce-494d-4419-a557-6f21d7534fd3 admin admin] PATH_INFO: `/v3/users/8b8fb3ca05db4960821c9f060d4cf032` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:03.093918 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de67bce-494d-4419-a557-6f21d7534fd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b8fb3ca05db4960821c9f060d4cf032)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:03.095941 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de67bce-494d-4419-a557-6f21d7534fd3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:03.308018 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6de67bce-494d-4419-a557-6f21d7534fd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b8fb3ca05db4960821c9f060d4cf032'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:03.381439 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6de67bce-494d-4419-a557-6f21d7534fd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b8fb3ca05db4960821c9f060d4cf032'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:03.395699 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6de67bce-494d-4419-a557-6f21d7534fd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b8fb3ca05db4960821c9f060d4cf032'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:03.396678 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6de67bce-494d-4419-a557-6f21d7534fd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b8fb3ca05db4960821c9f060d4cf032'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:03.404231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6de67bce-494d-4419-a557-6f21d7534fd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b8fb3ca05db4960821c9f060d4cf032'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:03.410331 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1389/2809] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:09:03 2022] DELETE /identity/v3/users/8b8fb3ca05db4960821c9f060d4cf032 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:03.853784 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39243abb-20ea-4ec8-b487-2f20f736b9a0 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:03.855427 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-39243abb-20ea-4ec8-b487-2f20f736b9a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:03.879642 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39243abb-20ea-4ec8-b487-2f20f736b9a0 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:03.963393 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:03.971616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989dcbf-a06d-40b7-9f47-f5fe85a314c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:03.972578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989dcbf-a06d-40b7-9f47-f5fe85a314c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:03.972970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989dcbf-a06d-40b7-9f47-f5fe85a314c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:03.973259 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989dcbf-a06d-40b7-9f47-f5fe85a314c2 admin admin] PATH_INFO: `/v3/projects/ba2384b621474b73ae6bd8ad9e3054e8` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:03.974497 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989dcbf-a06d-40b7-9f47-f5fe85a314c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba2384b621474b73ae6bd8ad9e3054e8)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:03.976481 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9989dcbf-a06d-40b7-9f47-f5fe85a314c2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:04.095844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9989dcbf-a06d-40b7-9f47-f5fe85a314c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba2384b621474b73ae6bd8ad9e3054e8', 'request_id': 'req-9989dcbf-a06d-40b7-9f47-f5fe85a314c2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:04.180953 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9989dcbf-a06d-40b7-9f47-f5fe85a314c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba2384b621474b73ae6bd8ad9e3054e8', 'request_id': 'req-9989dcbf-a06d-40b7-9f47-f5fe85a314c2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:04.187842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9989dcbf-a06d-40b7-9f47-f5fe85a314c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba2384b621474b73ae6bd8ad9e3054e8', 'request_id': 'req-9989dcbf-a06d-40b7-9f47-f5fe85a314c2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:04.198866 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9989dcbf-a06d-40b7-9f47-f5fe85a314c2 admin admin] The token cache is being invalidate because project ba2384b621474b73ae6bd8ad9e3054e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:09:04.199189 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9989dcbf-a06d-40b7-9f47-f5fe85a314c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:04.199857 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9989dcbf-a06d-40b7-9f47-f5fe85a314c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:04.201423 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1421/2810] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:09:03 2022] DELETE /identity/v3/projects/ba2384b621474b73ae6bd8ad9e3054e8 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:04.207608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d3a87-76c0-4491-9954-1573480fe2b2 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:04.208103 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d3a87-76c0-4491-9954-1573480fe2b2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:04.208206 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d3a87-76c0-4491-9954-1573480fe2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:04.216703 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:04.229830 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-296d3a87-76c0-4491-9954-1573480fe2b2 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:04.231067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-296d3a87-76c0-4491-9954-1573480fe2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:04.291894 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1390/2811] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:04.302635 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45566b-8685-4ea1-8da3-77aaa7db41cf None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:04.302993 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45566b-8685-4ea1-8da3-77aaa7db41cf None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:04.303233 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45566b-8685-4ea1-8da3-77aaa7db41cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:04.322983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-be45566b-8685-4ea1-8da3-77aaa7db41cf None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:04.324292 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-be45566b-8685-4ea1-8da3-77aaa7db41cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:04.332276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1422/2812] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:04.336781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d3a87-76c0-4491-9954-1573480fe2b2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:04.347202 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d3a87-76c0-4491-9954-1573480fe2b2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:04.351929 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2cc4d-45de-4633-bb22-47903e28e9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:04.357859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2cc4d-45de-4633-bb22-47903e28e9ab admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:04.357859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2cc4d-45de-4633-bb22-47903e28e9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:04.357859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2cc4d-45de-4633-bb22-47903e28e9ab admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:04.357859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2cc4d-45de-4633-bb22-47903e28e9ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:09:04.357859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2cc4d-45de-4633-bb22-47903e28e9ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:04.357859 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2cc4d-45de-4633-bb22-47903e28e9ab admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:04.367687 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1391/2813] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:09:04.372180 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45566b-8685-4ea1-8da3-77aaa7db41cf None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:04.381589 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45566b-8685-4ea1-8da3-77aaa7db41cf None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:04.386277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b0737-94d2-4802-9ee5-0f2f93714233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:04.387013 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b0737-94d2-4802-9ee5-0f2f93714233 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:04.387308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b0737-94d2-4802-9ee5-0f2f93714233 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:04.387581 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b0737-94d2-4802-9ee5-0f2f93714233 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:04.390639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b0737-94d2-4802-9ee5-0f2f93714233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:04.390639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32b0737-94d2-4802-9ee5-0f2f93714233 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:04.495979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1423/2814] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:09:04 2022] POST /identity/v3/projects => generated 391 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:09:04.500154 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2cc4d-45de-4633-bb22-47903e28e9ab admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:04.510141 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2cc4d-45de-4633-bb22-47903e28e9ab admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:04.581231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139ba3d-d8f4-4c1d-832d-a197c0db726b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:04.581843 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139ba3d-d8f4-4c1d-832d-a197c0db726b admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:04.582120 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139ba3d-d8f4-4c1d-832d-a197c0db726b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:04.582391 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139ba3d-d8f4-4c1d-832d-a197c0db726b admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:04.583067 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139ba3d-d8f4-4c1d-832d-a197c0db726b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:04.585494 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139ba3d-d8f4-4c1d-832d-a197c0db726b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:04.684436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1392/2815] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:09:04 2022] POST /identity/v3/users => generated 296 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:09:04.690332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b0737-94d2-4802-9ee5-0f2f93714233 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:04.701219 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32b0737-94d2-4802-9ee5-0f2f93714233 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:04.705813 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fabe49-c544-42ef-93fe-150059c02f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:04.706350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fabe49-c544-42ef-93fe-150059c02f77 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:04.706652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fabe49-c544-42ef-93fe-150059c02f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:04.706942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fabe49-c544-42ef-93fe-150059c02f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:04.707578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fabe49-c544-42ef-93fe-150059c02f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:04.709176 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fabe49-c544-42ef-93fe-150059c02f77 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:04.722167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1424/2816] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:04.727276 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139ba3d-d8f4-4c1d-832d-a197c0db726b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:04.738420 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139ba3d-d8f4-4c1d-832d-a197c0db726b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:04.747527 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697a771-9d8a-4335-ba2b-7ef8c280f5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:04.748219 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697a771-9d8a-4335-ba2b-7ef8c280f5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:04.748500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697a771-9d8a-4335-ba2b-7ef8c280f5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:04.748828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697a771-9d8a-4335-ba2b-7ef8c280f5b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:04.749542 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697a771-9d8a-4335-ba2b-7ef8c280f5b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:04.751396 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697a771-9d8a-4335-ba2b-7ef8c280f5b0 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:04.766952 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1393/2817] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:04.768985 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fabe49-c544-42ef-93fe-150059c02f77 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:04.780459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fabe49-c544-42ef-93fe-150059c02f77 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:04.785081 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd11ea4-ea20-43bb-bd82-d1542f8f46fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:04.785732 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd11ea4-ea20-43bb-bd82-d1542f8f46fd admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:04.786012 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd11ea4-ea20-43bb-bd82-d1542f8f46fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:04.786289 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd11ea4-ea20-43bb-bd82-d1542f8f46fd admin admin] PATH_INFO: `/v3/projects/ee8f68d2bff1414fa262c8beb34594dd/users/bf7076a19c4143848cacc23aa74faab7/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:04.800805 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd11ea4-ea20-43bb-bd82-d1542f8f46fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8f68d2bff1414fa262c8beb34594dd, user_id=bf7076a19c4143848cacc23aa74faab7, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:04.803534 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd11ea4-ea20-43bb-bd82-d1542f8f46fd admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:04.919260 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1425/2818] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:09:04 2022] PUT /identity/v3/projects/ee8f68d2bff1414fa262c8beb34594dd/users/bf7076a19c4143848cacc23aa74faab7/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:09:04.940062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981a082-db4f-4f22-9da2-c4aec13e9cb6 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:04.940349 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981a082-db4f-4f22-9da2-c4aec13e9cb6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:04.940621 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981a082-db4f-4f22-9da2-c4aec13e9cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:04.971606 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b981a082-db4f-4f22-9da2-c4aec13e9cb6 None None] MFA Rules not processed for user `bf7076a19c4143848cacc23aa74faab7`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:04.972650 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b981a082-db4f-4f22-9da2-c4aec13e9cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:05.042550 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:05.048035 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1394/2819] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:05.052670 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd11ea4-ea20-43bb-bd82-d1542f8f46fd admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:05.124713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd11ea4-ea20-43bb-bd82-d1542f8f46fd admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:05.130214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1419e58-b82d-4c52-a59a-db20361931d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:05.130829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1419e58-b82d-4c52-a59a-db20361931d8 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:05.131165 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1419e58-b82d-4c52-a59a-db20361931d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:05.131440 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1419e58-b82d-4c52-a59a-db20361931d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:05.132080 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1419e58-b82d-4c52-a59a-db20361931d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:05.134013 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1419e58-b82d-4c52-a59a-db20361931d8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:05.245389 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1426/2820] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:09:05 2022] POST /identity/v3/projects => generated 391 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:09:05.250132 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981a082-db4f-4f22-9da2-c4aec13e9cb6 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:05.260912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981a082-db4f-4f22-9da2-c4aec13e9cb6 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:05.318709 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54833d53-3385-4669-bb2c-313992b4535e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:05.319267 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54833d53-3385-4669-bb2c-313992b4535e admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:05.319549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54833d53-3385-4669-bb2c-313992b4535e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:05.319915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54833d53-3385-4669-bb2c-313992b4535e admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:05.320644 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54833d53-3385-4669-bb2c-313992b4535e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:05.323163 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54833d53-3385-4669-bb2c-313992b4535e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:05.419400 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1395/2821] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:09:05 2022] POST /identity/v3/users => generated 302 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:09:05.423679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1419e58-b82d-4c52-a59a-db20361931d8 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:05.437435 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1419e58-b82d-4c52-a59a-db20361931d8 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:05.442647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ae0db-4e45-426d-956a-e9993107aae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:05.443285 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ae0db-4e45-426d-956a-e9993107aae4 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:05.443571 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ae0db-4e45-426d-956a-e9993107aae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:05.443850 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ae0db-4e45-426d-956a-e9993107aae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:05.444489 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ae0db-4e45-426d-956a-e9993107aae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:05.446093 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ae0db-4e45-426d-956a-e9993107aae4 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:05.447566 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:05.460023 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1427/2822] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:05.464385 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54833d53-3385-4669-bb2c-313992b4535e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:05.475969 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54833d53-3385-4669-bb2c-313992b4535e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:05.480792 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dabcd0-1ce4-4dd8-8ce5-7717fcab3963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:05.482028 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dabcd0-1ce4-4dd8-8ce5-7717fcab3963 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:05.482304 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dabcd0-1ce4-4dd8-8ce5-7717fcab3963 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:05.483112 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dabcd0-1ce4-4dd8-8ce5-7717fcab3963 admin admin] PATH_INFO: `/v3/projects/78b0cdd3eb7142029aa48ee8b0dd42f6/users/0d3cd0ad433a4bbcb1511bba81db43b2/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:05.503420 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dabcd0-1ce4-4dd8-8ce5-7717fcab3963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78b0cdd3eb7142029aa48ee8b0dd42f6, user_id=0d3cd0ad433a4bbcb1511bba81db43b2, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:05.506069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dabcd0-1ce4-4dd8-8ce5-7717fcab3963 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:05.570598 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1396/2823] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:09:05 2022] PUT /identity/v3/projects/78b0cdd3eb7142029aa48ee8b0dd42f6/users/0d3cd0ad433a4bbcb1511bba81db43b2/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:05.574581 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ae0db-4e45-426d-956a-e9993107aae4 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:05.584652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ae0db-4e45-426d-956a-e9993107aae4 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:05.648265 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec52a14-3214-4ee5-a870-fcfb8c25322a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:05.648873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec52a14-3214-4ee5-a870-fcfb8c25322a admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:05.649146 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec52a14-3214-4ee5-a870-fcfb8c25322a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:05.649412 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec52a14-3214-4ee5-a870-fcfb8c25322a admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:05.650045 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec52a14-3214-4ee5-a870-fcfb8c25322a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:05.651787 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec52a14-3214-4ee5-a870-fcfb8c25322a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:05.665563 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1428/2824] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:05.670225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dabcd0-1ce4-4dd8-8ce5-7717fcab3963 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:05.685862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dabcd0-1ce4-4dd8-8ce5-7717fcab3963 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:05.690708 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702031c8-9938-4d89-b46b-b899e62366c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:05.691476 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702031c8-9938-4d89-b46b-b899e62366c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:05.691820 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702031c8-9938-4d89-b46b-b899e62366c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:05.692162 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702031c8-9938-4d89-b46b-b899e62366c5 admin admin] PATH_INFO: `/v3/projects/78b0cdd3eb7142029aa48ee8b0dd42f6/users/0d3cd0ad433a4bbcb1511bba81db43b2/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:05.694143 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702031c8-9938-4d89-b46b-b899e62366c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78b0cdd3eb7142029aa48ee8b0dd42f6, user_id=0d3cd0ad433a4bbcb1511bba81db43b2, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:05.696895 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702031c8-9938-4d89-b46b-b899e62366c5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:05.708121 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1397/2825] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:09:05 2022] PUT /identity/v3/projects/78b0cdd3eb7142029aa48ee8b0dd42f6/users/0d3cd0ad433a4bbcb1511bba81db43b2/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:05.719819 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b30fc0-43d8-48cf-a205-19af2d02c046 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:05.720338 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b30fc0-43d8-48cf-a205-19af2d02c046 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:05.720613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b30fc0-43d8-48cf-a205-19af2d02c046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:05.751686 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-34b30fc0-43d8-48cf-a205-19af2d02c046 None None] MFA Rules not processed for user `0d3cd0ad433a4bbcb1511bba81db43b2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:05.752847 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-34b30fc0-43d8-48cf-a205-19af2d02c046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:05.823052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1429/2826] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:05 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:09:05.835767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702031c8-9938-4d89-b46b-b899e62366c5 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:05.837609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-702031c8-9938-4d89-b46b-b899e62366c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:05.900426 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702031c8-9938-4d89-b46b-b899e62366c5 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:05.904996 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e095f5-5999-47c9-8431-e96db584e557 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:05.905848 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e095f5-5999-47c9-8431-e96db584e557 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:05.906119 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e095f5-5999-47c9-8431-e96db584e557 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:05.906408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e095f5-5999-47c9-8431-e96db584e557 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:05.916256 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e095f5-5999-47c9-8431-e96db584e557 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:05.920487 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e095f5-5999-47c9-8431-e96db584e557 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:05.927821 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1398/2827] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:05 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:05.975701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b30fc0-43d8-48cf-a205-19af2d02c046 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:05.977113 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-34b30fc0-43d8-48cf-a205-19af2d02c046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:05.990451 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b30fc0-43d8-48cf-a205-19af2d02c046 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:05.994150 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1db83-37d7-4a01-a75c-e1e070750e83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:05.994765 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1db83-37d7-4a01-a75c-e1e070750e83 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:05.995107 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1db83-37d7-4a01-a75c-e1e070750e83 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:05.995355 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1db83-37d7-4a01-a75c-e1e070750e83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:06.053929 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1db83-37d7-4a01-a75c-e1e070750e83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:06.055492 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1db83-37d7-4a01-a75c-e1e070750e83 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:06.066043 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1430/2828] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:10.751409 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e095f5-5999-47c9-8431-e96db584e557 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:10.802575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:10.813946 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e095f5-5999-47c9-8431-e96db584e557 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:10.818442 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce353c-d140-4ff0-b3b8-a06a9ca868ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:10.819106 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce353c-d140-4ff0-b3b8-a06a9ca868ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:10.819385 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce353c-d140-4ff0-b3b8-a06a9ca868ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:10.819655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce353c-d140-4ff0-b3b8-a06a9ca868ee admin admin] PATH_INFO: `/v3/users/bf7076a19c4143848cacc23aa74faab7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:10.820778 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce353c-d140-4ff0-b3b8-a06a9ca868ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf7076a19c4143848cacc23aa74faab7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:10.822714 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce353c-d140-4ff0-b3b8-a06a9ca868ee admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:11.046111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-39ce353c-d140-4ff0-b3b8-a06a9ca868ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf7076a19c4143848cacc23aa74faab7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:11.106769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-39ce353c-d140-4ff0-b3b8-a06a9ca868ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf7076a19c4143848cacc23aa74faab7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:11.120439 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-39ce353c-d140-4ff0-b3b8-a06a9ca868ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf7076a19c4143848cacc23aa74faab7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:11.121273 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-39ce353c-d140-4ff0-b3b8-a06a9ca868ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf7076a19c4143848cacc23aa74faab7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:11.129629 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-39ce353c-d140-4ff0-b3b8-a06a9ca868ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf7076a19c4143848cacc23aa74faab7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:11.136564 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1399/2829] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:09:10 2022] DELETE /identity/v3/users/bf7076a19c4143848cacc23aa74faab7 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:11.141616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adb2f6-cbdc-4e37-962c-75fb39b590a8 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:11.142122 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adb2f6-cbdc-4e37-962c-75fb39b590a8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:11.142224 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adb2f6-cbdc-4e37-962c-75fb39b590a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:11.161769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-11adb2f6-cbdc-4e37-962c-75fb39b590a8 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:11.162981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-11adb2f6-cbdc-4e37-962c-75fb39b590a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:11.173135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:11.224345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1431/2830] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:09:11.235044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce353c-d140-4ff0-b3b8-a06a9ca868ee admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:11.236928 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-39ce353c-d140-4ff0-b3b8-a06a9ca868ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:11.302548 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce353c-d140-4ff0-b3b8-a06a9ca868ee admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:11.306283 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe66450-599d-40f5-9dca-f13a777c35df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:11.306915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe66450-599d-40f5-9dca-f13a777c35df service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:11.307205 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe66450-599d-40f5-9dca-f13a777c35df service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:11.307478 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe66450-599d-40f5-9dca-f13a777c35df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:11.317438 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe66450-599d-40f5-9dca-f13a777c35df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:11.319005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe66450-599d-40f5-9dca-f13a777c35df service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:11.328639 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1400/2831] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:11.789085 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adb2f6-cbdc-4e37-962c-75fb39b590a8 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:11.791930 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-11adb2f6-cbdc-4e37-962c-75fb39b590a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:11.804624 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adb2f6-cbdc-4e37-962c-75fb39b590a8 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:11.812181 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9c871-9ae6-455f-bc58-b506106dbc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:11.812961 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9c871-9ae6-455f-bc58-b506106dbc99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:11.813299 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9c871-9ae6-455f-bc58-b506106dbc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:11.813756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9c871-9ae6-455f-bc58-b506106dbc99 admin admin] PATH_INFO: `/v3/projects/ee8f68d2bff1414fa262c8beb34594dd` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:11.815016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9c871-9ae6-455f-bc58-b506106dbc99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee8f68d2bff1414fa262c8beb34594dd)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:11.818655 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9c871-9ae6-455f-bc58-b506106dbc99 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:12.004982 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-11f9c871-9ae6-455f-bc58-b506106dbc99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee8f68d2bff1414fa262c8beb34594dd', 'request_id': 'req-11f9c871-9ae6-455f-bc58-b506106dbc99'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:12.078630 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-11f9c871-9ae6-455f-bc58-b506106dbc99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee8f68d2bff1414fa262c8beb34594dd', 'request_id': 'req-11f9c871-9ae6-455f-bc58-b506106dbc99'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:12.089459 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-11f9c871-9ae6-455f-bc58-b506106dbc99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee8f68d2bff1414fa262c8beb34594dd', 'request_id': 'req-11f9c871-9ae6-455f-bc58-b506106dbc99'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:12.115564 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-11f9c871-9ae6-455f-bc58-b506106dbc99 admin admin] The token cache is being invalidate because project ee8f68d2bff1414fa262c8beb34594dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:09:12.115860 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-11f9c871-9ae6-455f-bc58-b506106dbc99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:12.116543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-11f9c871-9ae6-455f-bc58-b506106dbc99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:12.118036 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1432/2832] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:09:11 2022] DELETE /identity/v3/projects/ee8f68d2bff1414fa262c8beb34594dd => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:12.122149 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe66450-599d-40f5-9dca-f13a777c35df service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:12.133236 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe66450-599d-40f5-9dca-f13a777c35df service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:12.200684 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e931062-aea2-49e0-808e-770f1e781e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:12.201322 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e931062-aea2-49e0-808e-770f1e781e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:12.201592 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e931062-aea2-49e0-808e-770f1e781e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:12.202355 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e931062-aea2-49e0-808e-770f1e781e15 admin admin] PATH_INFO: `/v3/users/0d3cd0ad433a4bbcb1511bba81db43b2` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:12.203530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e931062-aea2-49e0-808e-770f1e781e15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d3cd0ad433a4bbcb1511bba81db43b2)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:12.205445 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e931062-aea2-49e0-808e-770f1e781e15 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:12.223292 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:12.604277 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3e931062-aea2-49e0-808e-770f1e781e15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d3cd0ad433a4bbcb1511bba81db43b2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:12.689451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3e931062-aea2-49e0-808e-770f1e781e15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d3cd0ad433a4bbcb1511bba81db43b2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:12.704397 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3e931062-aea2-49e0-808e-770f1e781e15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d3cd0ad433a4bbcb1511bba81db43b2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:12.705516 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3e931062-aea2-49e0-808e-770f1e781e15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d3cd0ad433a4bbcb1511bba81db43b2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:12.714062 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3e931062-aea2-49e0-808e-770f1e781e15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d3cd0ad433a4bbcb1511bba81db43b2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:12.720564 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1401/2833] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:09:12 2022] DELETE /identity/v3/users/0d3cd0ad433a4bbcb1511bba81db43b2 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:13.199353 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9c871-9ae6-455f-bc58-b506106dbc99 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:13.200823 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-11f9c871-9ae6-455f-bc58-b506106dbc99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:13.217097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9c871-9ae6-455f-bc58-b506106dbc99 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:13.287512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b9917-3877-43bb-bad6-6c24776da260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:13.288349 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b9917-3877-43bb-bad6-6c24776da260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:13.288628 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b9917-3877-43bb-bad6-6c24776da260 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:13.288899 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b9917-3877-43bb-bad6-6c24776da260 admin admin] PATH_INFO: `/v3/projects/78b0cdd3eb7142029aa48ee8b0dd42f6` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:13.290047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b9917-3877-43bb-bad6-6c24776da260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78b0cdd3eb7142029aa48ee8b0dd42f6)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:13.292168 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b9917-3877-43bb-bad6-6c24776da260 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:13.388264 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:13.395051 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c55b9917-3877-43bb-bad6-6c24776da260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78b0cdd3eb7142029aa48ee8b0dd42f6', 'request_id': 'req-c55b9917-3877-43bb-bad6-6c24776da260'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:13.489243 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c55b9917-3877-43bb-bad6-6c24776da260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78b0cdd3eb7142029aa48ee8b0dd42f6', 'request_id': 'req-c55b9917-3877-43bb-bad6-6c24776da260'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:13.499318 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c55b9917-3877-43bb-bad6-6c24776da260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78b0cdd3eb7142029aa48ee8b0dd42f6', 'request_id': 'req-c55b9917-3877-43bb-bad6-6c24776da260'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:13.507546 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c55b9917-3877-43bb-bad6-6c24776da260 admin admin] The token cache is being invalidate because project 78b0cdd3eb7142029aa48ee8b0dd42f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:09:13.507833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c55b9917-3877-43bb-bad6-6c24776da260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:13.508492 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c55b9917-3877-43bb-bad6-6c24776da260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:13.509950 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1433/2834] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:09:13 2022] DELETE /identity/v3/projects/78b0cdd3eb7142029aa48ee8b0dd42f6 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:13.515866 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa4330-674a-4ed8-b02e-69e158f3c387 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:13.516456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa4330-674a-4ed8-b02e-69e158f3c387 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:13.516456 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa4330-674a-4ed8-b02e-69e158f3c387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:13.539004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-dbfa4330-674a-4ed8-b02e-69e158f3c387 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:13.540341 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-dbfa4330-674a-4ed8-b02e-69e158f3c387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:13.599918 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1402/2835] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:13.608488 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015149f6-9d7b-4ed8-b549-348d8f7dba49 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:13.608770 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015149f6-9d7b-4ed8-b549-348d8f7dba49 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:13.609043 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015149f6-9d7b-4ed8-b549-348d8f7dba49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:13.629548 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-015149f6-9d7b-4ed8-b549-348d8f7dba49 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:13.630763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-015149f6-9d7b-4ed8-b549-348d8f7dba49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:13.638655 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1434/2836] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:13.643004 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa4330-674a-4ed8-b02e-69e158f3c387 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:13.653322 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa4330-674a-4ed8-b02e-69e158f3c387 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:13.658044 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793f79d-e154-41cc-bf2c-9f3ab30f326c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:13.658668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793f79d-e154-41cc-bf2c-9f3ab30f326c admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:13.658997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793f79d-e154-41cc-bf2c-9f3ab30f326c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:13.659284 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793f79d-e154-41cc-bf2c-9f3ab30f326c admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:13.659927 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793f79d-e154-41cc-bf2c-9f3ab30f326c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:09:13.660226 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793f79d-e154-41cc-bf2c-9f3ab30f326c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:13.661818 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793f79d-e154-41cc-bf2c-9f3ab30f326c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:13.674214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1403/2837] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:09:13.678540 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015149f6-9d7b-4ed8-b549-348d8f7dba49 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:13.689373 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015149f6-9d7b-4ed8-b549-348d8f7dba49 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:13.694068 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d42a7b-3ded-4979-849b-96da33275cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:13.694673 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d42a7b-3ded-4979-849b-96da33275cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:13.695062 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d42a7b-3ded-4979-849b-96da33275cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:13.695502 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d42a7b-3ded-4979-849b-96da33275cc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:13.696135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d42a7b-3ded-4979-849b-96da33275cc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:13.698035 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d42a7b-3ded-4979-849b-96da33275cc0 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:13.786295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1435/2838] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:09:13 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:09:13.790349 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793f79d-e154-41cc-bf2c-9f3ab30f326c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:13.800175 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793f79d-e154-41cc-bf2c-9f3ab30f326c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:13.856180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0f378-9044-456c-ae7b-92d59978c90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:13.856778 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0f378-9044-456c-ae7b-92d59978c90c admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:13.857057 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0f378-9044-456c-ae7b-92d59978c90c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:13.857328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0f378-9044-456c-ae7b-92d59978c90c admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:13.857963 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0f378-9044-456c-ae7b-92d59978c90c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:13.859902 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0f378-9044-456c-ae7b-92d59978c90c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:13.944542 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1404/2839] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:09:13 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:09:13.949099 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d42a7b-3ded-4979-849b-96da33275cc0 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:13.960754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d42a7b-3ded-4979-849b-96da33275cc0 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:13.967191 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518d177-7891-45a4-90a0-08ba2c237a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:13.967881 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518d177-7891-45a4-90a0-08ba2c237a01 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:13.968160 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518d177-7891-45a4-90a0-08ba2c237a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:13.968428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518d177-7891-45a4-90a0-08ba2c237a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:13.969065 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518d177-7891-45a4-90a0-08ba2c237a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:13.970733 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518d177-7891-45a4-90a0-08ba2c237a01 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:13.981332 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1436/2840] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:13.985724 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0f378-9044-456c-ae7b-92d59978c90c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:13.995812 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0f378-9044-456c-ae7b-92d59978c90c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:14.000790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba21444-d640-4cf9-9fb8-fd1bf64ea4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:14.001385 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba21444-d640-4cf9-9fb8-fd1bf64ea4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:14.001659 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba21444-d640-4cf9-9fb8-fd1bf64ea4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:14.001932 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba21444-d640-4cf9-9fb8-fd1bf64ea4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:14.002567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba21444-d640-4cf9-9fb8-fd1bf64ea4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:14.004257 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba21444-d640-4cf9-9fb8-fd1bf64ea4f6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:14.009611 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:14.015949 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1405/2841] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:14.020296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518d177-7891-45a4-90a0-08ba2c237a01 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:14.030703 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518d177-7891-45a4-90a0-08ba2c237a01 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:14.036101 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5efa0-b8f1-4f46-b3d4-529d45b70866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:14.036756 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5efa0-b8f1-4f46-b3d4-529d45b70866 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:14.037037 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5efa0-b8f1-4f46-b3d4-529d45b70866 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:14.037313 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5efa0-b8f1-4f46-b3d4-529d45b70866 admin admin] PATH_INFO: `/v3/projects/a9c511a4bb4a4ffdb9aafe1f84bdcaa2/users/500f4ba8f0e547be965f08ce0ea7707b/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:14.053135 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5efa0-b8f1-4f46-b3d4-529d45b70866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9c511a4bb4a4ffdb9aafe1f84bdcaa2, user_id=500f4ba8f0e547be965f08ce0ea7707b, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:14.055867 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5efa0-b8f1-4f46-b3d4-529d45b70866 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:14.142357 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1437/2842] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:09:14 2022] PUT /identity/v3/projects/a9c511a4bb4a4ffdb9aafe1f84bdcaa2/users/500f4ba8f0e547be965f08ce0ea7707b/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:14.153097 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f1929-9c37-4d16-8e76-e14415a9e87a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:14.153384 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f1929-9c37-4d16-8e76-e14415a9e87a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:14.153661 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f1929-9c37-4d16-8e76-e14415a9e87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:14.184842 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f32f1929-9c37-4d16-8e76-e14415a9e87a None None] MFA Rules not processed for user `500f4ba8f0e547be965f08ce0ea7707b`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:14.186574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f32f1929-9c37-4d16-8e76-e14415a9e87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:14.262186 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1406/2843] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:14.274543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5efa0-b8f1-4f46-b3d4-529d45b70866 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:14.276267 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-94f5efa0-b8f1-4f46-b3d4-529d45b70866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:14.332948 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5efa0-b8f1-4f46-b3d4-529d45b70866 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:14.336611 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34701a9-e056-4e2d-bf94-f60c7de5c7cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:14.337211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34701a9-e056-4e2d-bf94-f60c7de5c7cf service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:14.337491 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34701a9-e056-4e2d-bf94-f60c7de5c7cf service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:14.337763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34701a9-e056-4e2d-bf94-f60c7de5c7cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:14.346000 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34701a9-e056-4e2d-bf94-f60c7de5c7cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:14.347465 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34701a9-e056-4e2d-bf94-f60c7de5c7cf service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:14.356192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1438/2844] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:14.376608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f1929-9c37-4d16-8e76-e14415a9e87a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:14.437860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f1929-9c37-4d16-8e76-e14415a9e87a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:14.442450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73e40c-59d7-440a-9e52-968a777ed2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:14.443094 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73e40c-59d7-440a-9e52-968a777ed2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:14.443374 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73e40c-59d7-440a-9e52-968a777ed2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:14.443647 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73e40c-59d7-440a-9e52-968a777ed2e8 admin admin] PATH_INFO: `/v3/users/500f4ba8f0e547be965f08ce0ea7707b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:14.444769 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73e40c-59d7-440a-9e52-968a777ed2e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=500f4ba8f0e547be965f08ce0ea7707b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:14.446655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73e40c-59d7-440a-9e52-968a777ed2e8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:14.604786 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5a73e40c-59d7-440a-9e52-968a777ed2e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '500f4ba8f0e547be965f08ce0ea7707b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:14.681137 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5a73e40c-59d7-440a-9e52-968a777ed2e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '500f4ba8f0e547be965f08ce0ea7707b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:14.694845 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5a73e40c-59d7-440a-9e52-968a777ed2e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '500f4ba8f0e547be965f08ce0ea7707b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:14.695530 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5a73e40c-59d7-440a-9e52-968a777ed2e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '500f4ba8f0e547be965f08ce0ea7707b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:14.703213 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5a73e40c-59d7-440a-9e52-968a777ed2e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '500f4ba8f0e547be965f08ce0ea7707b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:14.710964 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1407/2845] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:09:14 2022] DELETE /identity/v3/users/500f4ba8f0e547be965f08ce0ea7707b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:14.715399 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4aff7-fa4a-40de-88de-4c5440096000 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:14.715690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4aff7-fa4a-40de-88de-4c5440096000 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:14.716050 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4aff7-fa4a-40de-88de-4c5440096000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:14.735437 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f4c4aff7-fa4a-40de-88de-4c5440096000 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:14.736613 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f4c4aff7-fa4a-40de-88de-4c5440096000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:14.802229 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1439/2846] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:14.812473 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73e40c-59d7-440a-9e52-968a777ed2e8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:14.813869 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5a73e40c-59d7-440a-9e52-968a777ed2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:14.825006 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:14.872058 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73e40c-59d7-440a-9e52-968a777ed2e8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:14.876149 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede8859-915b-4366-b4e4-90a6a13849a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:14.876767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede8859-915b-4366-b4e4-90a6a13849a2 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:14.877254 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede8859-915b-4366-b4e4-90a6a13849a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:14.877585 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede8859-915b-4366-b4e4-90a6a13849a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:14.886780 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede8859-915b-4366-b4e4-90a6a13849a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:14.888249 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede8859-915b-4366-b4e4-90a6a13849a2 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:14.898602 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1408/2847] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:15.296622 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4aff7-fa4a-40de-88de-4c5440096000 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:15.299831 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f4c4aff7-fa4a-40de-88de-4c5440096000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:15.310617 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4aff7-fa4a-40de-88de-4c5440096000 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:15.317945 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9f94f-fbac-4594-9f8c-535b09880137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:15.318591 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9f94f-fbac-4594-9f8c-535b09880137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:15.318876 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9f94f-fbac-4594-9f8c-535b09880137 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:15.319187 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9f94f-fbac-4594-9f8c-535b09880137 admin admin] PATH_INFO: `/v3/projects/a9c511a4bb4a4ffdb9aafe1f84bdcaa2` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:15.320401 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9f94f-fbac-4594-9f8c-535b09880137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9c511a4bb4a4ffdb9aafe1f84bdcaa2)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:15.322441 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9f94f-fbac-4594-9f8c-535b09880137 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:15.324961 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:15.471721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-42d9f94f-fbac-4594-9f8c-535b09880137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9c511a4bb4a4ffdb9aafe1f84bdcaa2', 'request_id': 'req-42d9f94f-fbac-4594-9f8c-535b09880137'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:15.555245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-42d9f94f-fbac-4594-9f8c-535b09880137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9c511a4bb4a4ffdb9aafe1f84bdcaa2', 'request_id': 'req-42d9f94f-fbac-4594-9f8c-535b09880137'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:15.562660 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-42d9f94f-fbac-4594-9f8c-535b09880137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9c511a4bb4a4ffdb9aafe1f84bdcaa2', 'request_id': 'req-42d9f94f-fbac-4594-9f8c-535b09880137'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:15.573641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-42d9f94f-fbac-4594-9f8c-535b09880137 admin admin] The token cache is being invalidate because project a9c511a4bb4a4ffdb9aafe1f84bdcaa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:09:15.573926 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-42d9f94f-fbac-4594-9f8c-535b09880137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:15.574557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-42d9f94f-fbac-4594-9f8c-535b09880137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:15.576115 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1440/2848] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:09:15 2022] DELETE /identity/v3/projects/a9c511a4bb4a4ffdb9aafe1f84bdcaa2 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:15.581611 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ed3a3-a46a-49c3-aeee-87c983ae726f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:15.581896 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ed3a3-a46a-49c3-aeee-87c983ae726f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:15.582165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ed3a3-a46a-49c3-aeee-87c983ae726f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:15.601697 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-453ed3a3-a46a-49c3-aeee-87c983ae726f None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:15.602912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-453ed3a3-a46a-49c3-aeee-87c983ae726f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:15.663883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1409/2849] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:15.672866 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b5c11-fc0d-486a-8a73-9a6cfc71657b None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:15.674557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b5c11-fc0d-486a-8a73-9a6cfc71657b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:15.674557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b5c11-fc0d-486a-8a73-9a6cfc71657b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:15.695033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-612b5c11-fc0d-486a-8a73-9a6cfc71657b None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:15.696274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-612b5c11-fc0d-486a-8a73-9a6cfc71657b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:15.705211 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1441/2850] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:15.710767 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ed3a3-a46a-49c3-aeee-87c983ae726f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:15.722955 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ed3a3-a46a-49c3-aeee-87c983ae726f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:15.731373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6b320-1317-4f9c-9ed1-7481ad747352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:15.732086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6b320-1317-4f9c-9ed1-7481ad747352 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:15.732437 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6b320-1317-4f9c-9ed1-7481ad747352 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:15.732783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6b320-1317-4f9c-9ed1-7481ad747352 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:15.735489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6b320-1317-4f9c-9ed1-7481ad747352 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:09:15.735880 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6b320-1317-4f9c-9ed1-7481ad747352 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:15.737591 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6b320-1317-4f9c-9ed1-7481ad747352 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:15.753768 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1410/2851] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:09:15.755674 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b5c11-fc0d-486a-8a73-9a6cfc71657b None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:15.765776 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b5c11-fc0d-486a-8a73-9a6cfc71657b None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:15.771472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2e170-1b1d-41cb-8ceb-3e5360314377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:15.772206 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2e170-1b1d-41cb-8ceb-3e5360314377 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:15.772482 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2e170-1b1d-41cb-8ceb-3e5360314377 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:15.772748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2e170-1b1d-41cb-8ceb-3e5360314377 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:15.773407 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2e170-1b1d-41cb-8ceb-3e5360314377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:15.776097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2e170-1b1d-41cb-8ceb-3e5360314377 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:15.897565 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1442/2852] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:09:15 2022] POST /identity/v3/projects => generated 389 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:09:15.901826 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6b320-1317-4f9c-9ed1-7481ad747352 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:15.912644 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6b320-1317-4f9c-9ed1-7481ad747352 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:15.970161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdaf7ee-3ddb-4031-82c9-497a8436f776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:15.970860 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdaf7ee-3ddb-4031-82c9-497a8436f776 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:15.971225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdaf7ee-3ddb-4031-82c9-497a8436f776 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:15.971503 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdaf7ee-3ddb-4031-82c9-497a8436f776 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:15.972145 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdaf7ee-3ddb-4031-82c9-497a8436f776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:15.974066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdaf7ee-3ddb-4031-82c9-497a8436f776 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:16.069817 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1411/2853] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:09:15 2022] POST /identity/v3/users => generated 295 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:09:16.073907 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2e170-1b1d-41cb-8ceb-3e5360314377 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:16.085190 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2e170-1b1d-41cb-8ceb-3e5360314377 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:16.089842 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f78fa6-c3fa-49c6-83db-e8b15e393b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:16.090432 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f78fa6-c3fa-49c6-83db-e8b15e393b6d admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:16.090710 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f78fa6-c3fa-49c6-83db-e8b15e393b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:16.091020 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f78fa6-c3fa-49c6-83db-e8b15e393b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:16.091686 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f78fa6-c3fa-49c6-83db-e8b15e393b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:16.093350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f78fa6-c3fa-49c6-83db-e8b15e393b6d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:16.103390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1443/2854] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:16.107910 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdaf7ee-3ddb-4031-82c9-497a8436f776 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:16.118221 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdaf7ee-3ddb-4031-82c9-497a8436f776 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:16.122912 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d34801-d362-423a-ac84-ceb1bdf11661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:16.124013 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d34801-d362-423a-ac84-ceb1bdf11661 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:16.124291 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d34801-d362-423a-ac84-ceb1bdf11661 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:16.124562 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d34801-d362-423a-ac84-ceb1bdf11661 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:16.125198 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d34801-d362-423a-ac84-ceb1bdf11661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:16.126764 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d34801-d362-423a-ac84-ceb1bdf11661 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:16.138320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1412/2855] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:16.143383 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f78fa6-c3fa-49c6-83db-e8b15e393b6d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:16.153543 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f78fa6-c3fa-49c6-83db-e8b15e393b6d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:16.158467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5def86-316e-4067-9ed1-bb9288cc8dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:16.159145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5def86-316e-4067-9ed1-bb9288cc8dd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:16.159422 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5def86-316e-4067-9ed1-bb9288cc8dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:16.159694 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5def86-316e-4067-9ed1-bb9288cc8dd0 admin admin] PATH_INFO: `/v3/projects/6174b4e9749a4b4598e6f7f221ac6d55/users/5e47ef52d5f64ced9024dac2c6177882/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:16.174701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5def86-316e-4067-9ed1-bb9288cc8dd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6174b4e9749a4b4598e6f7f221ac6d55, user_id=5e47ef52d5f64ced9024dac2c6177882, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:16.177430 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5def86-316e-4067-9ed1-bb9288cc8dd0 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:16.232187 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1444/2856] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:09:16 2022] PUT /identity/v3/projects/6174b4e9749a4b4598e6f7f221ac6d55/users/5e47ef52d5f64ced9024dac2c6177882/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:09:16.299812 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ca88c-7702-44cc-abdd-e1214b89f6ff None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:16.300102 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ca88c-7702-44cc-abdd-e1214b89f6ff None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:16.300379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ca88c-7702-44cc-abdd-e1214b89f6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:16.331089 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-fb5ca88c-7702-44cc-abdd-e1214b89f6ff None None] MFA Rules not processed for user `5e47ef52d5f64ced9024dac2c6177882`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:16.332303 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fb5ca88c-7702-44cc-abdd-e1214b89f6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:16.337420 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:16.397216 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1413/2857] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:16.408774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5def86-316e-4067-9ed1-bb9288cc8dd0 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:16.410625 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9a5def86-316e-4067-9ed1-bb9288cc8dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:16.469774 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5def86-316e-4067-9ed1-bb9288cc8dd0 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:16.473647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815856f-9c12-49e7-b8ab-1b9f4311d166 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:16.474250 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815856f-9c12-49e7-b8ab-1b9f4311d166 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:16.474528 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815856f-9c12-49e7-b8ab-1b9f4311d166 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:16.474814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815856f-9c12-49e7-b8ab-1b9f4311d166 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:16.486007 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815856f-9c12-49e7-b8ab-1b9f4311d166 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:16.486918 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815856f-9c12-49e7-b8ab-1b9f4311d166 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:16.501043 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1445/2858] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:26.412759 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=24 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:26.493631 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:39.800140 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ca88c-7702-44cc-abdd-e1214b89f6ff None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:39.801816 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fb5ca88c-7702-44cc-abdd-e1214b89f6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:39.870808 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ca88c-7702-44cc-abdd-e1214b89f6ff None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:39.876207 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a353f-26eb-41ac-adec-f33eb96c862d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:39.876856 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a353f-26eb-41ac-adec-f33eb96c862d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:39.877135 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a353f-26eb-41ac-adec-f33eb96c862d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:39.877408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a353f-26eb-41ac-adec-f33eb96c862d admin admin] PATH_INFO: `/v3/users/ec2583029bc94c208ffa8502e38e4f3a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:39.878529 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a353f-26eb-41ac-adec-f33eb96c862d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec2583029bc94c208ffa8502e38e4f3a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:39.880487 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a353f-26eb-41ac-adec-f33eb96c862d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:39.981336 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815856f-9c12-49e7-b8ab-1b9f4311d166 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:39.992616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815856f-9c12-49e7-b8ab-1b9f4311d166 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:39.997297 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a6c64-4f2a-4f9d-a705-3264bc25d3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:39.998632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a6c64-4f2a-4f9d-a705-3264bc25d3f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:40.000901 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a6c64-4f2a-4f9d-a705-3264bc25d3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:40.000901 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a6c64-4f2a-4f9d-a705-3264bc25d3f6 admin admin] PATH_INFO: `/v3/users/5e47ef52d5f64ced9024dac2c6177882` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:40.000901 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a6c64-4f2a-4f9d-a705-3264bc25d3f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e47ef52d5f64ced9024dac2c6177882)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:40.003136 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a6c64-4f2a-4f9d-a705-3264bc25d3f6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:40.085838 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-174a353f-26eb-41ac-adec-f33eb96c862d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec2583029bc94c208ffa8502e38e4f3a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:40.154884 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-174a353f-26eb-41ac-adec-f33eb96c862d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec2583029bc94c208ffa8502e38e4f3a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:40.168666 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-174a353f-26eb-41ac-adec-f33eb96c862d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec2583029bc94c208ffa8502e38e4f3a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:40.169984 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-174a353f-26eb-41ac-adec-f33eb96c862d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec2583029bc94c208ffa8502e38e4f3a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:40.177241 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-174a353f-26eb-41ac-adec-f33eb96c862d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec2583029bc94c208ffa8502e38e4f3a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:40.183965 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1414/2859] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:09:39 2022] DELETE /identity/v3/users/ec2583029bc94c208ffa8502e38e4f3a => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:40.188627 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05530995-fd7c-4c76-b3df-2685d08dfc97 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:40.188917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05530995-fd7c-4c76-b3df-2685d08dfc97 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:40.192000 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05530995-fd7c-4c76-b3df-2685d08dfc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:40.209286 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-05530995-fd7c-4c76-b3df-2685d08dfc97 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:40.210429 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-05530995-fd7c-4c76-b3df-2685d08dfc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:40.237512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-458a6c64-4f2a-4f9d-a705-3264bc25d3f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e47ef52d5f64ced9024dac2c6177882'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:40.272350 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1415/2860] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:09:40.283706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05530995-fd7c-4c76-b3df-2685d08dfc97 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:40.285074 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-05530995-fd7c-4c76-b3df-2685d08dfc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:40.307575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:40.320394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-458a6c64-4f2a-4f9d-a705-3264bc25d3f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e47ef52d5f64ced9024dac2c6177882'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:40.321127 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-458a6c64-4f2a-4f9d-a705-3264bc25d3f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e47ef52d5f64ced9024dac2c6177882'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:40.329097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-458a6c64-4f2a-4f9d-a705-3264bc25d3f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e47ef52d5f64ced9024dac2c6177882'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:40.342871 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-458a6c64-4f2a-4f9d-a705-3264bc25d3f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e47ef52d5f64ced9024dac2c6177882'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:40.345853 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05530995-fd7c-4c76-b3df-2685d08dfc97 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:40.349385 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1446/2861] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:09:39 2022] DELETE /identity/v3/users/5e47ef52d5f64ced9024dac2c6177882 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:09:40.349778 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac16d3-43a3-4df0-8140-35f90f4d84eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:40.350408 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac16d3-43a3-4df0-8140-35f90f4d84eb service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:40.350685 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac16d3-43a3-4df0-8140-35f90f4d84eb service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:40.350987 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac16d3-43a3-4df0-8140-35f90f4d84eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:40.352827 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-baac16d3-43a3-4df0-8140-35f90f4d84eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:40.354103 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72df29-e300-499b-b9f5-abdf6173ba03 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:40.354388 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72df29-e300-499b-b9f5-abdf6173ba03 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:40.354663 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72df29-e300-499b-b9f5-abdf6173ba03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:40.366765 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac16d3-43a3-4df0-8140-35f90f4d84eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:40.368419 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac16d3-43a3-4df0-8140-35f90f4d84eb service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:40.375067 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3f72df29-e300-499b-b9f5-abdf6173ba03 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:40.376368 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3f72df29-e300-499b-b9f5-abdf6173ba03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:40.379016 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1416/2862] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:09:40.438727 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1447/2863] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:09:40.455037 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac16d3-43a3-4df0-8140-35f90f4d84eb service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:40.456971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-baac16d3-43a3-4df0-8140-35f90f4d84eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:40.472427 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac16d3-43a3-4df0-8140-35f90f4d84eb service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:40.533713 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482aee3f-86ca-433a-9817-31238aec7ac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:40.534307 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aee3f-86ca-433a-9817-31238aec7ac6 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:40.534584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aee3f-86ca-433a-9817-31238aec7ac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:40.534883 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482aee3f-86ca-433a-9817-31238aec7ac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:40.544410 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482aee3f-86ca-433a-9817-31238aec7ac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:40.546213 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482aee3f-86ca-433a-9817-31238aec7ac6 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:40.555778 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1417/2864] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:40.875470 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72df29-e300-499b-b9f5-abdf6173ba03 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:40.877228 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3f72df29-e300-499b-b9f5-abdf6173ba03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:40.892058 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72df29-e300-499b-b9f5-abdf6173ba03 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:40.897420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d1dc47-4572-460f-b41d-6013a2e1c0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:40.900995 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1dc47-4572-460f-b41d-6013a2e1c0ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:40.900995 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1dc47-4572-460f-b41d-6013a2e1c0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:40.900995 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1dc47-4572-460f-b41d-6013a2e1c0ab admin admin] PATH_INFO: `/v3/projects/30b7603d3bf84c8fbfa5b77ff92b9347` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:40.900995 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1dc47-4572-460f-b41d-6013a2e1c0ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=30b7603d3bf84c8fbfa5b77ff92b9347)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:40.901997 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1dc47-4572-460f-b41d-6013a2e1c0ab admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:41.042616 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-50d1dc47-4572-460f-b41d-6013a2e1c0ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30b7603d3bf84c8fbfa5b77ff92b9347', 'request_id': 'req-50d1dc47-4572-460f-b41d-6013a2e1c0ab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:41.060811 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482aee3f-86ca-433a-9817-31238aec7ac6 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:41.067225 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-482aee3f-86ca-433a-9817-31238aec7ac6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:41.090118 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482aee3f-86ca-433a-9817-31238aec7ac6 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:41.148632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-50d1dc47-4572-460f-b41d-6013a2e1c0ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30b7603d3bf84c8fbfa5b77ff92b9347', 'request_id': 'req-50d1dc47-4572-460f-b41d-6013a2e1c0ab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:41.150568 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41123f06-b800-417d-8ae7-73a3b411d25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:41.151276 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41123f06-b800-417d-8ae7-73a3b411d25a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:41.151558 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41123f06-b800-417d-8ae7-73a3b411d25a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:41.151961 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41123f06-b800-417d-8ae7-73a3b411d25a admin admin] PATH_INFO: `/v3/projects/6174b4e9749a4b4598e6f7f221ac6d55` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:41.153091 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41123f06-b800-417d-8ae7-73a3b411d25a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6174b4e9749a4b4598e6f7f221ac6d55)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:41.155191 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41123f06-b800-417d-8ae7-73a3b411d25a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:41.155679 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-50d1dc47-4572-460f-b41d-6013a2e1c0ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30b7603d3bf84c8fbfa5b77ff92b9347', 'request_id': 'req-50d1dc47-4572-460f-b41d-6013a2e1c0ab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:41.160991 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:41.167782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-50d1dc47-4572-460f-b41d-6013a2e1c0ab admin admin] The token cache is being invalidate because project 30b7603d3bf84c8fbfa5b77ff92b9347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:09:41.168075 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-50d1dc47-4572-460f-b41d-6013a2e1c0ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:41.168742 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-50d1dc47-4572-460f-b41d-6013a2e1c0ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:41.170302 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1448/2865] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:09:40 2022] DELETE /identity/v3/projects/30b7603d3bf84c8fbfa5b77ff92b9347 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:41.274070 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-41123f06-b800-417d-8ae7-73a3b411d25a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6174b4e9749a4b4598e6f7f221ac6d55', 'request_id': 'req-41123f06-b800-417d-8ae7-73a3b411d25a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:41.280738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-41123f06-b800-417d-8ae7-73a3b411d25a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6174b4e9749a4b4598e6f7f221ac6d55', 'request_id': 'req-41123f06-b800-417d-8ae7-73a3b411d25a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:41.287466 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-41123f06-b800-417d-8ae7-73a3b411d25a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6174b4e9749a4b4598e6f7f221ac6d55', 'request_id': 'req-41123f06-b800-417d-8ae7-73a3b411d25a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:41.291740 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:41.379157 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-41123f06-b800-417d-8ae7-73a3b411d25a admin admin] The token cache is being invalidate because project 6174b4e9749a4b4598e6f7f221ac6d55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:09:41.387549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-41123f06-b800-417d-8ae7-73a3b411d25a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:41.387549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-41123f06-b800-417d-8ae7-73a3b411d25a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:41.387549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1418/2866] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:09:41 2022] DELETE /identity/v3/projects/6174b4e9749a4b4598e6f7f221ac6d55 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:09:41.387884 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796bb3c1-ebaf-4c5f-b3c3-badb07f11cb1 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:41.387884 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796bb3c1-ebaf-4c5f-b3c3-badb07f11cb1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:41.387884 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796bb3c1-ebaf-4c5f-b3c3-badb07f11cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:41.411016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-796bb3c1-ebaf-4c5f-b3c3-badb07f11cb1 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:41.416276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-796bb3c1-ebaf-4c5f-b3c3-badb07f11cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:41.485201 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1449/2867] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:41.493326 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2834b-f26d-4eb5-acd7-5c5df01c8239 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:41.493604 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2834b-f26d-4eb5-acd7-5c5df01c8239 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:41.493886 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2834b-f26d-4eb5-acd7-5c5df01c8239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:41.515536 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-12d2834b-f26d-4eb5-acd7-5c5df01c8239 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:41.516763 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-12d2834b-f26d-4eb5-acd7-5c5df01c8239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:41.527180 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1419/2868] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:41.531800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796bb3c1-ebaf-4c5f-b3c3-badb07f11cb1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:41.542398 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796bb3c1-ebaf-4c5f-b3c3-badb07f11cb1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:41.547680 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365c2f8-9635-43af-a945-fb72eb037860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:41.548310 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365c2f8-9635-43af-a945-fb72eb037860 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:41.548596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365c2f8-9635-43af-a945-fb72eb037860 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:41.548942 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365c2f8-9635-43af-a945-fb72eb037860 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:41.549511 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365c2f8-9635-43af-a945-fb72eb037860 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:09:41.549811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365c2f8-9635-43af-a945-fb72eb037860 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:41.551456 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365c2f8-9635-43af-a945-fb72eb037860 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:41.566683 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1450/2869] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:09:41.568994 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2834b-f26d-4eb5-acd7-5c5df01c8239 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:41.578838 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2834b-f26d-4eb5-acd7-5c5df01c8239 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:41.584051 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92080c-e1ff-47c6-9d3a-15c36b1fe6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:41.587968 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92080c-e1ff-47c6-9d3a-15c36b1fe6f9 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:41.587968 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92080c-e1ff-47c6-9d3a-15c36b1fe6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:41.587968 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92080c-e1ff-47c6-9d3a-15c36b1fe6f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:41.587968 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92080c-e1ff-47c6-9d3a-15c36b1fe6f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:41.587968 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92080c-e1ff-47c6-9d3a-15c36b1fe6f9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:41.698913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1420/2870] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:09:41 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:09:41.703237 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365c2f8-9635-43af-a945-fb72eb037860 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:41.713397 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365c2f8-9635-43af-a945-fb72eb037860 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:41.770547 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe82cb-1741-4fcb-a4f7-f5513b568670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:41.771165 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe82cb-1741-4fcb-a4f7-f5513b568670 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:41.771448 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe82cb-1741-4fcb-a4f7-f5513b568670 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:41.771721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe82cb-1741-4fcb-a4f7-f5513b568670 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:41.772362 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe82cb-1741-4fcb-a4f7-f5513b568670 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:41.774279 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe82cb-1741-4fcb-a4f7-f5513b568670 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:41.903449 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1451/2871] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:09:41 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:09:41.908734 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92080c-e1ff-47c6-9d3a-15c36b1fe6f9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:41.920781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92080c-e1ff-47c6-9d3a-15c36b1fe6f9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:41.925532 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904907eb-ed3a-4daf-8010-f68b501614b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:41.926140 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904907eb-ed3a-4daf-8010-f68b501614b6 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:41.926438 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904907eb-ed3a-4daf-8010-f68b501614b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:41.926717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904907eb-ed3a-4daf-8010-f68b501614b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:41.927399 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904907eb-ed3a-4daf-8010-f68b501614b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:41.929049 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904907eb-ed3a-4daf-8010-f68b501614b6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:41.939575 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1421/2872] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:09:41.943700 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe82cb-1741-4fcb-a4f7-f5513b568670 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:41.954433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe82cb-1741-4fcb-a4f7-f5513b568670 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:41.958975 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6a281-f37f-4269-aec5-94a9799a120d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:41.959598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6a281-f37f-4269-aec5-94a9799a120d admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:41.959875 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6a281-f37f-4269-aec5-94a9799a120d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:41.960141 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6a281-f37f-4269-aec5-94a9799a120d admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:41.960767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6a281-f37f-4269-aec5-94a9799a120d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:41.962347 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6a281-f37f-4269-aec5-94a9799a120d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:41.973809 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1452/2873] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:41.978168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904907eb-ed3a-4daf-8010-f68b501614b6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:41.988389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904907eb-ed3a-4daf-8010-f68b501614b6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:41.992997 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c64c83-fabb-4b65-a9cf-cc083aaf46ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:41.993655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c64c83-fabb-4b65-a9cf-cc083aaf46ee admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:41.993931 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c64c83-fabb-4b65-a9cf-cc083aaf46ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:41.994201 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c64c83-fabb-4b65-a9cf-cc083aaf46ee admin admin] PATH_INFO: `/v3/projects/f7224a31a8aa4e7ca9514d77b5f00526/users/4b8776b82114444cbe35aa534fe14fa2/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:42.015264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c64c83-fabb-4b65-a9cf-cc083aaf46ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7224a31a8aa4e7ca9514d77b5f00526, user_id=4b8776b82114444cbe35aa534fe14fa2, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:42.018190 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c64c83-fabb-4b65-a9cf-cc083aaf46ee admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:42.111755 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1422/2874] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:09:41 2022] PUT /identity/v3/projects/f7224a31a8aa4e7ca9514d77b5f00526/users/4b8776b82114444cbe35aa534fe14fa2/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:42.117924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12fb53-3810-41df-89ab-0f113dd94be4 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:42.118404 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12fb53-3810-41df-89ab-0f113dd94be4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:42.118666 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12fb53-3810-41df-89ab-0f113dd94be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:42.139848 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ea12fb53-3810-41df-89ab-0f113dd94be4 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:42.143595 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ea12fb53-3810-41df-89ab-0f113dd94be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:42.215670 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1453/2875] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:42.225500 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c64c83-fabb-4b65-a9cf-cc083aaf46ee admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:42.227252 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d9c64c83-fabb-4b65-a9cf-cc083aaf46ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:42.286712 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c64c83-fabb-4b65-a9cf-cc083aaf46ee admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:42.291896 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6240d0d-5982-40f0-8cf2-d0da7e291556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:42.292609 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6240d0d-5982-40f0-8cf2-d0da7e291556 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:42.292888 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6240d0d-5982-40f0-8cf2-d0da7e291556 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:42.293161 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6240d0d-5982-40f0-8cf2-d0da7e291556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:42.302668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6240d0d-5982-40f0-8cf2-d0da7e291556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:42.304237 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6240d0d-5982-40f0-8cf2-d0da7e291556 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:42.314971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1423/2876] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:47.915134 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee7a65-728a-42d7-ade9-e883592029c1 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:47.915767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee7a65-728a-42d7-ade9-e883592029c1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:47.915767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee7a65-728a-42d7-ade9-e883592029c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:47.923052 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:47.951747 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-86ee7a65-728a-42d7-ade9-e883592029c1 None None] MFA Rules not processed for user `4b8776b82114444cbe35aa534fe14fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:47.953512 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-86ee7a65-728a-42d7-ade9-e883592029c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:48.028985 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1454/2877] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:09:48.033743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6240d0d-5982-40f0-8cf2-d0da7e291556 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:48.043837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6240d0d-5982-40f0-8cf2-d0da7e291556 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:48.049075 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990053e-8cad-4d35-b421-f42bbeb69d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:48.061168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990053e-8cad-4d35-b421-f42bbeb69d05 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:48.061168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990053e-8cad-4d35-b421-f42bbeb69d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:48.061168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990053e-8cad-4d35-b421-f42bbeb69d05 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:48.061168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990053e-8cad-4d35-b421-f42bbeb69d05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:48.061168 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990053e-8cad-4d35-b421-f42bbeb69d05 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:48.170944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1424/2878] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:09:48 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:09:48.178242 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee7a65-728a-42d7-ade9-e883592029c1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:48.193557 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee7a65-728a-42d7-ade9-e883592029c1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:48.272351 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27b9ad-b71f-42b0-a1a8-f5bcc19c59ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:48.272973 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27b9ad-b71f-42b0-a1a8-f5bcc19c59ab admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:48.273249 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27b9ad-b71f-42b0-a1a8-f5bcc19c59ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:48.273518 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27b9ad-b71f-42b0-a1a8-f5bcc19c59ab admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:48.274145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27b9ad-b71f-42b0-a1a8-f5bcc19c59ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:48.277080 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27b9ad-b71f-42b0-a1a8-f5bcc19c59ab admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:48.370450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1455/2879] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:09:48 2022] POST /identity/v3/users => generated 295 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:09:48.375086 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990053e-8cad-4d35-b421-f42bbeb69d05 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:48.388160 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990053e-8cad-4d35-b421-f42bbeb69d05 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:48.397245 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4bb4c-3c97-4341-ac81-524be13ee7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:48.398250 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4bb4c-3c97-4341-ac81-524be13ee7de admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:48.398722 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4bb4c-3c97-4341-ac81-524be13ee7de admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:48.399069 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4bb4c-3c97-4341-ac81-524be13ee7de admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:48.399831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4bb4c-3c97-4341-ac81-524be13ee7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:48.401572 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4bb4c-3c97-4341-ac81-524be13ee7de admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:48.418114 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1425/2880] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:48.420033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27b9ad-b71f-42b0-a1a8-f5bcc19c59ab admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:48.431604 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27b9ad-b71f-42b0-a1a8-f5bcc19c59ab admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:48.436882 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4805fac-547b-40dd-851a-a979a04d4e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:48.441376 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4805fac-547b-40dd-851a-a979a04d4e83 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:48.441655 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4805fac-547b-40dd-851a-a979a04d4e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:48.441933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4805fac-547b-40dd-851a-a979a04d4e83 admin admin] PATH_INFO: `/v3/projects/deac2c5d3b114aa88eb0426fbe824e05/users/336431a86fcc46c4b62f225d5abef579/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:48.459794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4805fac-547b-40dd-851a-a979a04d4e83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deac2c5d3b114aa88eb0426fbe824e05, user_id=336431a86fcc46c4b62f225d5abef579, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:48.462472 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4805fac-547b-40dd-851a-a979a04d4e83 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:48.557298 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1456/2881] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:09:48 2022] PUT /identity/v3/projects/deac2c5d3b114aa88eb0426fbe824e05/users/336431a86fcc46c4b62f225d5abef579/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:48.563174 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4bb4c-3c97-4341-ac81-524be13ee7de admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:48.574938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4bb4c-3c97-4341-ac81-524be13ee7de admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:48.646008 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:48.657300 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a612d4b-5d8b-4cfa-a866-4ce42acceb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:48.657917 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a612d4b-5d8b-4cfa-a866-4ce42acceb7d admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:48.658436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a612d4b-5d8b-4cfa-a866-4ce42acceb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:48.658436 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a612d4b-5d8b-4cfa-a866-4ce42acceb7d admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:48.659320 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a612d4b-5d8b-4cfa-a866-4ce42acceb7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:48.660944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a612d4b-5d8b-4cfa-a866-4ce42acceb7d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:48.677230 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1426/2882] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:48.678718 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4805fac-547b-40dd-851a-a979a04d4e83 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:48.690062 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4805fac-547b-40dd-851a-a979a04d4e83 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:48.695221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f22f1-5d70-4b56-b92b-94f00a71c05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:48.695934 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f22f1-5d70-4b56-b92b-94f00a71c05d admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:48.696209 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f22f1-5d70-4b56-b92b-94f00a71c05d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:48.696484 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f22f1-5d70-4b56-b92b-94f00a71c05d admin admin] PATH_INFO: `/v3/projects/deac2c5d3b114aa88eb0426fbe824e05/users/336431a86fcc46c4b62f225d5abef579/roles/32d91beb67f2452c995f72173bb8ea4f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:48.698496 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f22f1-5d70-4b56-b92b-94f00a71c05d admin admin] RBAC: Authorizing `identity:create_grant(project_id=deac2c5d3b114aa88eb0426fbe824e05, user_id=336431a86fcc46c4b62f225d5abef579, role_id=32d91beb67f2452c995f72173bb8ea4f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:48.701761 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f22f1-5d70-4b56-b92b-94f00a71c05d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:48.714314 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1457/2883] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:09:48 2022] PUT /identity/v3/projects/deac2c5d3b114aa88eb0426fbe824e05/users/336431a86fcc46c4b62f225d5abef579/roles/32d91beb67f2452c995f72173bb8ea4f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:54.463706 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b99be-ecc4-4733-9ffa-7f719ebd0051 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:54.464642 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b99be-ecc4-4733-9ffa-7f719ebd0051 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:54.465117 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b99be-ecc4-4733-9ffa-7f719ebd0051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:54.505095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-745b99be-ecc4-4733-9ffa-7f719ebd0051 None None] MFA Rules not processed for user `336431a86fcc46c4b62f225d5abef579`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:54.506610 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-745b99be-ecc4-4733-9ffa-7f719ebd0051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:54.585677 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1427/2884] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:09:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:54.603295 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f22f1-5d70-4b56-b92b-94f00a71c05d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:54.605591 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ac6f22f1-5d70-4b56-b92b-94f00a71c05d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:54.657004 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:54.687322 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f22f1-5d70-4b56-b92b-94f00a71c05d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:54.693450 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31235bfe-6bf5-42a3-9f75-96f1f89d0c73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:54.693944 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31235bfe-6bf5-42a3-9f75-96f1f89d0c73 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:54.694225 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31235bfe-6bf5-42a3-9f75-96f1f89d0c73 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:54.694499 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31235bfe-6bf5-42a3-9f75-96f1f89d0c73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:54.762455 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31235bfe-6bf5-42a3-9f75-96f1f89d0c73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:54.762455 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31235bfe-6bf5-42a3-9f75-96f1f89d0c73 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:54.768713 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1458/2885] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:59.243746 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b99be-ecc4-4733-9ffa-7f719ebd0051 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:59.245231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-745b99be-ecc4-4733-9ffa-7f719ebd0051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:59.306040 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b99be-ecc4-4733-9ffa-7f719ebd0051 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:59.310154 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae9597-7de1-4af1-97a3-df52e343767f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:59.315182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae9597-7de1-4af1-97a3-df52e343767f service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:59.315182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae9597-7de1-4af1-97a3-df52e343767f service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:59.315182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae9597-7de1-4af1-97a3-df52e343767f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:59.315182 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5cae9597-7de1-4af1-97a3-df52e343767f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:59.328361 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae9597-7de1-4af1-97a3-df52e343767f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:59.329893 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae9597-7de1-4af1-97a3-df52e343767f service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:59.341581 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1428/2886] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:03.426308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31235bfe-6bf5-42a3-9f75-96f1f89d0c73 service nova] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:03.482817 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31235bfe-6bf5-42a3-9f75-96f1f89d0c73 service nova] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:03.486647 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b339af-fa15-4d28-8e86-e8e1dd2ae5ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:03.487283 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b339af-fa15-4d28-8e86-e8e1dd2ae5ec service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:03.487556 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b339af-fa15-4d28-8e86-e8e1dd2ae5ec service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:03.487824 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b339af-fa15-4d28-8e86-e8e1dd2ae5ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:03.489607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a4b339af-fa15-4d28-8e86-e8e1dd2ae5ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:03.504894 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b339af-fa15-4d28-8e86-e8e1dd2ae5ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:03.506346 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b339af-fa15-4d28-8e86-e8e1dd2ae5ec service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:03.518083 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1459/2887] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:09.336425 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=65 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:13.512875 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=62 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:17.304970 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b058f-5362-48ed-8163-439c67e22434 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:17.305556 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b058f-5362-48ed-8163-439c67e22434 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:17.305556 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b058f-5362-48ed-8163-439c67e22434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:17.324725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-275b058f-5362-48ed-8163-439c67e22434 None None] MFA Rules not processed for user `4ac5fd494df14ee1b0faf745dcc8c1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:17.326001 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-275b058f-5362-48ed-8163-439c67e22434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:17.383153 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1429/2888] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:10:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:18.000530 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b339af-fa15-4d28-8e86-e8e1dd2ae5ec service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:18.003026 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a4b339af-fa15-4d28-8e86-e8e1dd2ae5ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:18.077241 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b339af-fa15-4d28-8e86-e8e1dd2ae5ec service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:18.082232 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0689b-1cc2-4bfc-ab36-6d54b2ad04d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9506139d5744fe69153e7e30a05b436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:18.082839 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0689b-1cc2-4bfc-ab36-6d54b2ad04d6 service placement] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:18.083154 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0689b-1cc2-4bfc-ab36-6d54b2ad04d6 service placement] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:18.083426 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0689b-1cc2-4bfc-ab36-6d54b2ad04d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:18.085162 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-40f0689b-1cc2-4bfc-ab36-6d54b2ad04d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:18.096802 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0689b-1cc2-4bfc-ab36-6d54b2ad04d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:18.099137 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0689b-1cc2-4bfc-ab36-6d54b2ad04d6 service placement] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:18.108608 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1460/2889] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:18.744212 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f97e3-eebf-48b6-b645-db108e41dfea None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:18.744745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f97e3-eebf-48b6-b645-db108e41dfea None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:18.744745 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f97e3-eebf-48b6-b645-db108e41dfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:18.764707 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-905f97e3-eebf-48b6-b645-db108e41dfea None None] MFA Rules not processed for user `4b8776b82114444cbe35aa534fe14fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:18.765923 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-905f97e3-eebf-48b6-b645-db108e41dfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:18.773060 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1430/2890] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:10:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:21.220690 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0689b-1cc2-4bfc-ab36-6d54b2ad04d6 service placement] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:21.223992 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-40f0689b-1cc2-4bfc-ab36-6d54b2ad04d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:21.234768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0689b-1cc2-4bfc-ab36-6d54b2ad04d6 service placement] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:21.238684 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab7d73-ac52-455b-b6fc-f880b20c365a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:21.239320 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab7d73-ac52-455b-b6fc-f880b20c365a service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:21.239596 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab7d73-ac52-455b-b6fc-f880b20c365a service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:21.239861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab7d73-ac52-455b-b6fc-f880b20c365a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:21.246392 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c6ab7d73-ac52-455b-b6fc-f880b20c365a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:21.257524 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab7d73-ac52-455b-b6fc-f880b20c365a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:21.265598 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab7d73-ac52-455b-b6fc-f880b20c365a service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:21.268437 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1461/2891] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:10:25.230376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f97e3-eebf-48b6-b645-db108e41dfea None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:25.231933 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-905f97e3-eebf-48b6-b645-db108e41dfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:25.248913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f97e3-eebf-48b6-b645-db108e41dfea None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:25.255539 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587aec6-7ecc-4df6-b497-acfd74fc055f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:25.256233 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587aec6-7ecc-4df6-b497-acfd74fc055f service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:25.256604 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587aec6-7ecc-4df6-b497-acfd74fc055f service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:25.256938 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587aec6-7ecc-4df6-b497-acfd74fc055f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:25.259956 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4587aec6-7ecc-4df6-b497-acfd74fc055f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:25.274508 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587aec6-7ecc-4df6-b497-acfd74fc055f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:25.276467 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587aec6-7ecc-4df6-b497-acfd74fc055f service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:25.288556 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1431/2892] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:31.263833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=38 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:35.283122 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=34 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:37.962114 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab7d73-ac52-455b-b6fc-f880b20c365a service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:37.970845 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab7d73-ac52-455b-b6fc-f880b20c365a service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:37.975485 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48235c-b8ba-4617-b7e1-c10d01b93b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:37.976350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48235c-b8ba-4617-b7e1-c10d01b93b22 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:37.976632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48235c-b8ba-4617-b7e1-c10d01b93b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:37.976902 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48235c-b8ba-4617-b7e1-c10d01b93b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:37.985970 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48235c-b8ba-4617-b7e1-c10d01b93b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:37.987447 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48235c-b8ba-4617-b7e1-c10d01b93b22 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:37.996921 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1462/2893] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:40.006863 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587aec6-7ecc-4df6-b497-acfd74fc055f service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:40.019431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587aec6-7ecc-4df6-b497-acfd74fc055f service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:40.025655 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17895e-5c02-45e1-aa78-96a56421b2e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:40.028009 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17895e-5c02-45e1-aa78-96a56421b2e1 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:40.028009 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17895e-5c02-45e1-aa78-96a56421b2e1 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:40.028009 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17895e-5c02-45e1-aa78-96a56421b2e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:40.028729 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2d17895e-5c02-45e1-aa78-96a56421b2e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:40.042080 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17895e-5c02-45e1-aa78-96a56421b2e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:40.043733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17895e-5c02-45e1-aa78-96a56421b2e1 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:40.055583 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1432/2894] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:47.992681 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=6 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:50.050366 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=6 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:53.439594 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48235c-b8ba-4617-b7e1-c10d01b93b22 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:53.448983 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48235c-b8ba-4617-b7e1-c10d01b93b22 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:53.452652 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dae27d-9ad5-4079-aa64-f21584844ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:53.453244 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dae27d-9ad5-4079-aa64-f21584844ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:53.453525 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dae27d-9ad5-4079-aa64-f21584844ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:53.453800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dae27d-9ad5-4079-aa64-f21584844ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:53.455604 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d4dae27d-9ad5-4079-aa64-f21584844ae1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:53.525782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dae27d-9ad5-4079-aa64-f21584844ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:53.527180 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dae27d-9ad5-4079-aa64-f21584844ae1 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:53.537403 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1463/2895] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:11:01.247993 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17895e-5c02-45e1-aa78-96a56421b2e1 service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:01.249906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2d17895e-5c02-45e1-aa78-96a56421b2e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:01.265031 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17895e-5c02-45e1-aa78-96a56421b2e1 service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:01.270415 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832c92f-ef15-47ed-a5fa-2bf0bf57de1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:01.271622 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832c92f-ef15-47ed-a5fa-2bf0bf57de1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:01.271903 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832c92f-ef15-47ed-a5fa-2bf0bf57de1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:01.272173 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832c92f-ef15-47ed-a5fa-2bf0bf57de1e admin admin] PATH_INFO: `/v3/users/30cfda5bb8284716935e7670bf44f647` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:01.273391 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832c92f-ef15-47ed-a5fa-2bf0bf57de1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=30cfda5bb8284716935e7670bf44f647)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:01.275420 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832c92f-ef15-47ed-a5fa-2bf0bf57de1e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:01.497937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3832c92f-ef15-47ed-a5fa-2bf0bf57de1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30cfda5bb8284716935e7670bf44f647'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:01.571988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3832c92f-ef15-47ed-a5fa-2bf0bf57de1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30cfda5bb8284716935e7670bf44f647'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:01.585194 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3832c92f-ef15-47ed-a5fa-2bf0bf57de1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30cfda5bb8284716935e7670bf44f647'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:01.585828 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3832c92f-ef15-47ed-a5fa-2bf0bf57de1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30cfda5bb8284716935e7670bf44f647'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:01.593256 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3832c92f-ef15-47ed-a5fa-2bf0bf57de1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30cfda5bb8284716935e7670bf44f647'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:01.600248 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1433/2896] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:11:01 2022] DELETE /identity/v3/users/30cfda5bb8284716935e7670bf44f647 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:01.888007 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dae27d-9ad5-4079-aa64-f21584844ae1 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:01.899676 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dae27d-9ad5-4079-aa64-f21584844ae1 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:01.964999 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52537666-25e4-470f-b253-154054f9f391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:01.965627 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52537666-25e4-470f-b253-154054f9f391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:01.965908 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52537666-25e4-470f-b253-154054f9f391 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:01.966184 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52537666-25e4-470f-b253-154054f9f391 admin admin] PATH_INFO: `/v3/projects/0d4361bb4c424b41aed6f6afa040df84` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:01.967319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52537666-25e4-470f-b253-154054f9f391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d4361bb4c424b41aed6f6afa040df84)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:01.969222 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52537666-25e4-470f-b253-154054f9f391 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:02.051611 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-52537666-25e4-470f-b253-154054f9f391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d4361bb4c424b41aed6f6afa040df84', 'request_id': 'req-52537666-25e4-470f-b253-154054f9f391'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:02.120526 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-52537666-25e4-470f-b253-154054f9f391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d4361bb4c424b41aed6f6afa040df84', 'request_id': 'req-52537666-25e4-470f-b253-154054f9f391'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:02.127335 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-52537666-25e4-470f-b253-154054f9f391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d4361bb4c424b41aed6f6afa040df84', 'request_id': 'req-52537666-25e4-470f-b253-154054f9f391'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:02.138387 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-52537666-25e4-470f-b253-154054f9f391 admin admin] The token cache is being invalidate because project 0d4361bb4c424b41aed6f6afa040df84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:11:02.138673 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-52537666-25e4-470f-b253-154054f9f391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:02.139319 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-52537666-25e4-470f-b253-154054f9f391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:02.140794 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1464/2897] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:11:01 2022] DELETE /identity/v3/projects/0d4361bb4c424b41aed6f6afa040df84 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:02.145209 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832c92f-ef15-47ed-a5fa-2bf0bf57de1e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:02.146649 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3832c92f-ef15-47ed-a5fa-2bf0bf57de1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:02.159992 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832c92f-ef15-47ed-a5fa-2bf0bf57de1e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:02.217103 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d522f4f-0b4d-4b72-acb4-66b92ac1f2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:02.217726 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d522f4f-0b4d-4b72-acb4-66b92ac1f2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:02.218009 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d522f4f-0b4d-4b72-acb4-66b92ac1f2be admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:02.218279 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d522f4f-0b4d-4b72-acb4-66b92ac1f2be admin admin] PATH_INFO: `/v3/users/dc7727d9b8864d7c9eb7f3ba02f245f2` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:02.219458 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d522f4f-0b4d-4b72-acb4-66b92ac1f2be admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc7727d9b8864d7c9eb7f3ba02f245f2)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:02.221378 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d522f4f-0b4d-4b72-acb4-66b92ac1f2be admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:02.471315 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2d522f4f-0b4d-4b72-acb4-66b92ac1f2be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc7727d9b8864d7c9eb7f3ba02f245f2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:02.542633 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2d522f4f-0b4d-4b72-acb4-66b92ac1f2be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc7727d9b8864d7c9eb7f3ba02f245f2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:02.555847 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2d522f4f-0b4d-4b72-acb4-66b92ac1f2be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc7727d9b8864d7c9eb7f3ba02f245f2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:02.556492 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2d522f4f-0b4d-4b72-acb4-66b92ac1f2be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc7727d9b8864d7c9eb7f3ba02f245f2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:02.558474 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:02.563628 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2d522f4f-0b4d-4b72-acb4-66b92ac1f2be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc7727d9b8864d7c9eb7f3ba02f245f2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:02.569344 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1434/2898] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:11:02 2022] DELETE /identity/v3/users/dc7727d9b8864d7c9eb7f3ba02f245f2 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:02.808131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52537666-25e4-470f-b253-154054f9f391 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:02.809578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-52537666-25e4-470f-b253-154054f9f391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:02.823467 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52537666-25e4-470f-b253-154054f9f391 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:02.847252 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:02.884558 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2a29c-bdc2-456f-b445-95d210b7b233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:02.885180 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2a29c-bdc2-456f-b445-95d210b7b233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:02.885454 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2a29c-bdc2-456f-b445-95d210b7b233 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:02.885721 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2a29c-bdc2-456f-b445-95d210b7b233 admin admin] PATH_INFO: `/v3/projects/b20af97c0ebd4b0684d8af91ff968997` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:02.887301 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2a29c-bdc2-456f-b445-95d210b7b233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b20af97c0ebd4b0684d8af91ff968997)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:02.889194 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2a29c-bdc2-456f-b445-95d210b7b233 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:03.006766 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b9a2a29c-bdc2-456f-b445-95d210b7b233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b20af97c0ebd4b0684d8af91ff968997', 'request_id': 'req-b9a2a29c-bdc2-456f-b445-95d210b7b233'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:03.051461 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b9a2a29c-bdc2-456f-b445-95d210b7b233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b20af97c0ebd4b0684d8af91ff968997', 'request_id': 'req-b9a2a29c-bdc2-456f-b445-95d210b7b233'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:03.057996 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b9a2a29c-bdc2-456f-b445-95d210b7b233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b20af97c0ebd4b0684d8af91ff968997', 'request_id': 'req-b9a2a29c-bdc2-456f-b445-95d210b7b233'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:03.069833 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b9a2a29c-bdc2-456f-b445-95d210b7b233 admin admin] The token cache is being invalidate because project b20af97c0ebd4b0684d8af91ff968997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:11:03.070121 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b9a2a29c-bdc2-456f-b445-95d210b7b233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:03.070763 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b9a2a29c-bdc2-456f-b445-95d210b7b233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:03.072282 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1465/2899] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:11:02 2022] DELETE /identity/v3/projects/b20af97c0ebd4b0684d8af91ff968997 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:12.685322 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:13.066309 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:25.634571 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d522f4f-0b4d-4b72-acb4-66b92ac1f2be admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:25.636516 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2d522f4f-0b4d-4b72-acb4-66b92ac1f2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:25.651380 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d522f4f-0b4d-4b72-acb4-66b92ac1f2be admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:25.707856 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b832-655e-4e92-9056-58d11912fecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:25.708480 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b832-655e-4e92-9056-58d11912fecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:25.708791 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b832-655e-4e92-9056-58d11912fecb admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:25.709066 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b832-655e-4e92-9056-58d11912fecb admin admin] PATH_INFO: `/v3/users/4ac5fd494df14ee1b0faf745dcc8c1e3` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:25.710184 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b832-655e-4e92-9056-58d11912fecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ac5fd494df14ee1b0faf745dcc8c1e3)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:25.712129 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b832-655e-4e92-9056-58d11912fecb admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:25.875050 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4fa4b832-655e-4e92-9056-58d11912fecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ac5fd494df14ee1b0faf745dcc8c1e3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:25.940203 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4fa4b832-655e-4e92-9056-58d11912fecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ac5fd494df14ee1b0faf745dcc8c1e3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:25.952967 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4fa4b832-655e-4e92-9056-58d11912fecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ac5fd494df14ee1b0faf745dcc8c1e3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:25.953652 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4fa4b832-655e-4e92-9056-58d11912fecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ac5fd494df14ee1b0faf745dcc8c1e3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:25.961238 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4fa4b832-655e-4e92-9056-58d11912fecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ac5fd494df14ee1b0faf745dcc8c1e3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:25.967209 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1435/2900] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:11:25 2022] DELETE /identity/v3/users/4ac5fd494df14ee1b0faf745dcc8c1e3 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:26.217027 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2a29c-bdc2-456f-b445-95d210b7b233 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:26.218544 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b9a2a29c-bdc2-456f-b445-95d210b7b233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:26.232958 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2a29c-bdc2-456f-b445-95d210b7b233 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:26.290650 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c84df7-4776-4989-ab79-d9ec7dc32cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:26.291308 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c84df7-4776-4989-ab79-d9ec7dc32cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:26.291590 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c84df7-4776-4989-ab79-d9ec7dc32cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:26.291864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c84df7-4776-4989-ab79-d9ec7dc32cfc admin admin] PATH_INFO: `/v3/projects/736f37b101ed43369000a5ba9d93e90f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:26.292981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c84df7-4776-4989-ab79-d9ec7dc32cfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=736f37b101ed43369000a5ba9d93e90f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:26.294884 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c84df7-4776-4989-ab79-d9ec7dc32cfc admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:26.432209 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-88c84df7-4776-4989-ab79-d9ec7dc32cfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '736f37b101ed43369000a5ba9d93e90f', 'request_id': 'req-88c84df7-4776-4989-ab79-d9ec7dc32cfc'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:26.508277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-88c84df7-4776-4989-ab79-d9ec7dc32cfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '736f37b101ed43369000a5ba9d93e90f', 'request_id': 'req-88c84df7-4776-4989-ab79-d9ec7dc32cfc'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:26.515116 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-88c84df7-4776-4989-ab79-d9ec7dc32cfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '736f37b101ed43369000a5ba9d93e90f', 'request_id': 'req-88c84df7-4776-4989-ab79-d9ec7dc32cfc'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:26.526782 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-88c84df7-4776-4989-ab79-d9ec7dc32cfc admin admin] The token cache is being invalidate because project 736f37b101ed43369000a5ba9d93e90f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:11:26.527104 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-88c84df7-4776-4989-ab79-d9ec7dc32cfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:26.527768 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-88c84df7-4776-4989-ab79-d9ec7dc32cfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:26.529334 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1466/2901] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:11:26 2022] DELETE /identity/v3/projects/736f37b101ed43369000a5ba9d93e90f => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:11:26.534219 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38604f0f-8c50-4c08-b2bc-2ac4767de314 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:26.534504 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38604f0f-8c50-4c08-b2bc-2ac4767de314 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:26.534781 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38604f0f-8c50-4c08-b2bc-2ac4767de314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:26.556119 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-38604f0f-8c50-4c08-b2bc-2ac4767de314 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:26.557325 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-38604f0f-8c50-4c08-b2bc-2ac4767de314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:26.620149 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1436/2902] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:26.628769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584266f3-4a7b-4bcf-9ab8-8c9755135062 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:26.629047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584266f3-4a7b-4bcf-9ab8-8c9755135062 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:26.629318 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584266f3-4a7b-4bcf-9ab8-8c9755135062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:26.650274 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-584266f3-4a7b-4bcf-9ab8-8c9755135062 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:26.651580 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-584266f3-4a7b-4bcf-9ab8-8c9755135062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:26.659428 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1467/2903] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:26.664103 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38604f0f-8c50-4c08-b2bc-2ac4767de314 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:26.675042 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38604f0f-8c50-4c08-b2bc-2ac4767de314 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:26.680711 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d8ba7-3389-4aa4-985c-6855f2e4600d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:26.681309 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d8ba7-3389-4aa4-985c-6855f2e4600d admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:26.681584 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d8ba7-3389-4aa4-985c-6855f2e4600d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:26.681850 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d8ba7-3389-4aa4-985c-6855f2e4600d admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:26.682482 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d8ba7-3389-4aa4-985c-6855f2e4600d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:11:26.682783 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d8ba7-3389-4aa4-985c-6855f2e4600d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:26.684440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d8ba7-3389-4aa4-985c-6855f2e4600d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:26.696915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1437/2904] 10.222.0.91 () {58 vars in 1153 bytes} [Thu Sep 8 14:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:11:26.701538 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584266f3-4a7b-4bcf-9ab8-8c9755135062 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:26.711193 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584266f3-4a7b-4bcf-9ab8-8c9755135062 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:26.715749 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75d96f-acc6-47f8-ae1b-aca880d0409e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:26.716333 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75d96f-acc6-47f8-ae1b-aca880d0409e admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:26.716607 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75d96f-acc6-47f8-ae1b-aca880d0409e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:26.716873 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75d96f-acc6-47f8-ae1b-aca880d0409e admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:26.717523 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75d96f-acc6-47f8-ae1b-aca880d0409e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:26.719434 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75d96f-acc6-47f8-ae1b-aca880d0409e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:26.818754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1468/2905] 10.222.0.91 () {60 vars in 1153 bytes} [Thu Sep 8 14:11:26 2022] POST /identity/v3/projects => generated 399 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:11:26.822797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d8ba7-3389-4aa4-985c-6855f2e4600d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:26.833146 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d8ba7-3389-4aa4-985c-6855f2e4600d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:26.836694 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:26.890411 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4000ceb-e619-46db-b75b-f8ed853d5d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:26.891038 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4000ceb-e619-46db-b75b-f8ed853d5d74 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:26.891314 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4000ceb-e619-46db-b75b-f8ed853d5d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:26.891583 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4000ceb-e619-46db-b75b-f8ed853d5d74 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:26.892463 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4000ceb-e619-46db-b75b-f8ed853d5d74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:26.894448 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4000ceb-e619-46db-b75b-f8ed853d5d74 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:26.977164 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1438/2906] 10.222.0.91 () {60 vars in 1144 bytes} [Thu Sep 8 14:11:26 2022] POST /identity/v3/users => generated 300 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:11:26.981296 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75d96f-acc6-47f8-ae1b-aca880d0409e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:26.992388 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75d96f-acc6-47f8-ae1b-aca880d0409e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:26.997036 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fcd2c-bf51-4fa0-93a9-cc0b8ad7c1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:26.997638 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fcd2c-bf51-4fa0-93a9-cc0b8ad7c1ff admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:26.997933 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fcd2c-bf51-4fa0-93a9-cc0b8ad7c1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:26.998208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fcd2c-bf51-4fa0-93a9-cc0b8ad7c1ff admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:26.998835 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160fcd2c-bf51-4fa0-93a9-cc0b8ad7c1ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:27.000471 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160fcd2c-bf51-4fa0-93a9-cc0b8ad7c1ff admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:27.011036 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1469/2907] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:11:27.023092 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4000ceb-e619-46db-b75b-f8ed853d5d74 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:27.033619 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4000ceb-e619-46db-b75b-f8ed853d5d74 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:27.038735 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec808cd-cadf-4902-904c-8b845253cd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:27.039608 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec808cd-cadf-4902-904c-8b845253cd45 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:27.039890 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec808cd-cadf-4902-904c-8b845253cd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:27.040158 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec808cd-cadf-4902-904c-8b845253cd45 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:27.040790 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec808cd-cadf-4902-904c-8b845253cd45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:27.042373 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec808cd-cadf-4902-904c-8b845253cd45 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:27.053648 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1439/2908] 10.222.0.91 () {58 vars in 1122 bytes} [Thu Sep 8 14:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:11:27.057795 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fcd2c-bf51-4fa0-93a9-cc0b8ad7c1ff admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:27.067754 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fcd2c-bf51-4fa0-93a9-cc0b8ad7c1ff admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:27.072394 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55874c-79e1-42e4-ad42-2f9592c9bbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:27.073047 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55874c-79e1-42e4-ad42-2f9592c9bbae admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:27.073345 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55874c-79e1-42e4-ad42-2f9592c9bbae admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:27.073599 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55874c-79e1-42e4-ad42-2f9592c9bbae admin admin] PATH_INFO: `/v3/projects/f978c658bc8c455c8750fc30d544e124/users/7382c7ae06794712918262392a36c514/roles/cb722313950945e5b147e994ec678cd1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:27.088018 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55874c-79e1-42e4-ad42-2f9592c9bbae admin admin] RBAC: Authorizing `identity:create_grant(project_id=f978c658bc8c455c8750fc30d544e124, user_id=7382c7ae06794712918262392a36c514, role_id=cb722313950945e5b147e994ec678cd1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:27.090661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55874c-79e1-42e4-ad42-2f9592c9bbae admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:27.193800 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1470/2909] 10.222.0.91 () {60 vars in 1483 bytes} [Thu Sep 8 14:11:27 2022] PUT /identity/v3/projects/f978c658bc8c455c8750fc30d544e124/users/7382c7ae06794712918262392a36c514/roles/cb722313950945e5b147e994ec678cd1 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:27.205431 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f9460-ca65-4e37-a18c-98cad40ce556 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:27.205717 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f9460-ca65-4e37-a18c-98cad40ce556 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:27.205985 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f9460-ca65-4e37-a18c-98cad40ce556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:27.233449 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e49f9460-ca65-4e37-a18c-98cad40ce556 None None] MFA Rules not processed for user `7382c7ae06794712918262392a36c514`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:27.237165 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e49f9460-ca65-4e37-a18c-98cad40ce556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:27.296950 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1440/2910] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:11:27 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:27.309356 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55874c-79e1-42e4-ad42-2f9592c9bbae admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:27.311070 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9d55874c-79e1-42e4-ad42-2f9592c9bbae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:27.341639 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:27.369659 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55874c-79e1-42e4-ad42-2f9592c9bbae admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:27.374781 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02187efa-76a9-4cf6-852d-8193627a0f09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca225b99584e40f4a4065bf10264974d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:27.375490 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02187efa-76a9-4cf6-852d-8193627a0f09 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:27.375748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02187efa-76a9-4cf6-852d-8193627a0f09 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:27.376019 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02187efa-76a9-4cf6-852d-8193627a0f09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:27.384740 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02187efa-76a9-4cf6-852d-8193627a0f09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:27.386170 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02187efa-76a9-4cf6-852d-8193627a0f09 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:27.395531 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1471/2911] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:27 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:11:35.235668 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f9460-ca65-4e37-a18c-98cad40ce556 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:35.297980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f9460-ca65-4e37-a18c-98cad40ce556 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:35.302459 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f584a-470f-48f4-a1dc-7a4b0689f7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:35.303111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f584a-470f-48f4-a1dc-7a4b0689f7ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:35.303391 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f584a-470f-48f4-a1dc-7a4b0689f7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:35.303664 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9f584a-470f-48f4-a1dc-7a4b0689f7ae admin admin] PATH_INFO: `/v3/users/7382c7ae06794712918262392a36c514` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:35.304831 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f584a-470f-48f4-a1dc-7a4b0689f7ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=7382c7ae06794712918262392a36c514)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:35.306738 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9f584a-470f-48f4-a1dc-7a4b0689f7ae admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:35.323451 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:35.467240 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2e9f584a-470f-48f4-a1dc-7a4b0689f7ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7382c7ae06794712918262392a36c514'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:35.559434 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2e9f584a-470f-48f4-a1dc-7a4b0689f7ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7382c7ae06794712918262392a36c514'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:35.573347 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2e9f584a-470f-48f4-a1dc-7a4b0689f7ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7382c7ae06794712918262392a36c514'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:35.574014 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2e9f584a-470f-48f4-a1dc-7a4b0689f7ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7382c7ae06794712918262392a36c514'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:35.581024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2e9f584a-470f-48f4-a1dc-7a4b0689f7ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7382c7ae06794712918262392a36c514'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:35.587574 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1441/2912] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:11:35 2022] DELETE /identity/v3/users/7382c7ae06794712918262392a36c514 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:35.592156 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f2f17-6fd5-40b2-a904-ba1ee8a7f8c7 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:35.592632 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f2f17-6fd5-40b2-a904-ba1ee8a7f8c7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:35.592748 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f2f17-6fd5-40b2-a904-ba1ee8a7f8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:35.611587 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ba1f2f17-6fd5-40b2-a904-ba1ee8a7f8c7 None None] MFA Rules not processed for user `c14eb0bf645e49f887a8d288d37d1491`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:35.612798 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ba1f2f17-6fd5-40b2-a904-ba1ee8a7f8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:35.671913 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1472/2913] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:35.682389 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f584a-470f-48f4-a1dc-7a4b0689f7ae admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:35.683906 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2e9f584a-470f-48f4-a1dc-7a4b0689f7ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:35.742525 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9f584a-470f-48f4-a1dc-7a4b0689f7ae admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:35.746024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e0c8c-46af-4182-82f2-d0bca42d4c6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:35.746624 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e0c8c-46af-4182-82f2-d0bca42d4c6f service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:35.746915 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e0c8c-46af-4182-82f2-d0bca42d4c6f service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:35.747195 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e0c8c-46af-4182-82f2-d0bca42d4c6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:35.756241 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e0c8c-46af-4182-82f2-d0bca42d4c6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:35.758285 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e0c8c-46af-4182-82f2-d0bca42d4c6f service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:35.768085 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1442/2914] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:11:36.223743 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f2f17-6fd5-40b2-a904-ba1ee8a7f8c7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:36.225231 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ba1f2f17-6fd5-40b2-a904-ba1ee8a7f8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:36.237506 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f2f17-6fd5-40b2-a904-ba1ee8a7f8c7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:36.242811 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:36.243475 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:36.243862 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:36.244028 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5 admin admin] PATH_INFO: `/v3/projects/f978c658bc8c455c8750fc30d544e124` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:36.245150 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f978c658bc8c455c8750fc30d544e124)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:36.247097 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:36.365987 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f978c658bc8c455c8750fc30d544e124', 'request_id': 'req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:36.438876 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f978c658bc8c455c8750fc30d544e124', 'request_id': 'req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:36.445698 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f978c658bc8c455c8750fc30d544e124', 'request_id': 'req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:36.456221 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5 admin admin] The token cache is being invalidate because project f978c658bc8c455c8750fc30d544e124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:11:36.456505 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:36.457131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:36.458633 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1473/2915] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:11:36 2022] DELETE /identity/v3/projects/f978c658bc8c455c8750fc30d544e124 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:45.763686 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=9,keystone:SELECT=46,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:46.453145 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=68,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:12:43.166815 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91567aef-4c26-43f8-80ec-a25f9b585576 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:43.167347 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91567aef-4c26-43f8-80ec-a25f9b585576 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:43.167489 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91567aef-4c26-43f8-80ec-a25f9b585576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:43.186702 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-91567aef-4c26-43f8-80ec-a25f9b585576 None None] MFA Rules not processed for user `4b8776b82114444cbe35aa534fe14fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:12:43.188034 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-91567aef-4c26-43f8-80ec-a25f9b585576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:43.243774 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1443/2916] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:12:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:12:43.563366 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:43.564855 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:43.623578 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c77c1-7230-4e5a-b85e-02b4c765d3c5 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:43.627245 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c7c90-4775-49a0-a201-9a61b3c58458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:43.627841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c7c90-4775-49a0-a201-9a61b3c58458 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:43.628118 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c7c90-4775-49a0-a201-9a61b3c58458 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:43.628390 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c7c90-4775-49a0-a201-9a61b3c58458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:43.630750 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ae7c7c90-4775-49a0-a201-9a61b3c58458 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:43.640656 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c7c90-4775-49a0-a201-9a61b3c58458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:43.642223 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c7c90-4775-49a0-a201-9a61b3c58458 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:43.651127 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1474/2917] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:12:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:12:53.236733 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=25 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:12:53.647790 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=26 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:12:57.789314 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91567aef-4c26-43f8-80ec-a25f9b585576 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:57.791005 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-91567aef-4c26-43f8-80ec-a25f9b585576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:57.864416 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91567aef-4c26-43f8-80ec-a25f9b585576 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:57.870810 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04187120-c4af-4cd6-b5c8-cdb6db4083bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:57.871534 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04187120-c4af-4cd6-b5c8-cdb6db4083bc service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:57.871937 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04187120-c4af-4cd6-b5c8-cdb6db4083bc service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:57.872214 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04187120-c4af-4cd6-b5c8-cdb6db4083bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:57.873971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-04187120-c4af-4cd6-b5c8-cdb6db4083bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:57.887803 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04187120-c4af-4cd6-b5c8-cdb6db4083bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:57.889432 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04187120-c4af-4cd6-b5c8-cdb6db4083bc service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:57.901704 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1444/2918] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:12:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:13:07.896473 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=34 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:05.738498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c7c90-4775-49a0-a201-9a61b3c58458 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:05.738498 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ae7c7c90-4775-49a0-a201-9a61b3c58458 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:05.808022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c7c90-4775-49a0-a201-9a61b3c58458 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:05.813208 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c84252-b9bf-4e35-8adb-df81a6143a93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9506139d5744fe69153e7e30a05b436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:05.818981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c84252-b9bf-4e35-8adb-df81a6143a93 service placement] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:05.818981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c84252-b9bf-4e35-8adb-df81a6143a93 service placement] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:05.818981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c84252-b9bf-4e35-8adb-df81a6143a93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:05.818981 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c9c84252-b9bf-4e35-8adb-df81a6143a93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:05.828686 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c84252-b9bf-4e35-8adb-df81a6143a93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:05.830266 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c84252-b9bf-4e35-8adb-df81a6143a93 service placement] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:05.840769 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1475/2919] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:14:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:14:15.836307 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=32 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:37.751567 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b15c8e-dee8-45a9-9bd9-1dd39871d943 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:37.752123 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b15c8e-dee8-45a9-9bd9-1dd39871d943 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:37.752123 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b15c8e-dee8-45a9-9bd9-1dd39871d943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:37.771484 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-51b15c8e-dee8-45a9-9bd9-1dd39871d943 None None] MFA Rules not processed for user `4b8776b82114444cbe35aa534fe14fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:37.772683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-51b15c8e-dee8-45a9-9bd9-1dd39871d943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:37.780099 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1445/2920] 10.222.0.91 () {58 vars in 958 bytes} [Thu Sep 8 14:14:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:14:47.766239 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:55.128778 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c84252-b9bf-4e35-8adb-df81a6143a93 service placement] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:55.130164 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c9c84252-b9bf-4e35-8adb-df81a6143a93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:55.146421 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c84252-b9bf-4e35-8adb-df81a6143a93 service placement] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:55.151924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a076cfd-8347-4c8d-aac4-93df3474505e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:55.152583 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a076cfd-8347-4c8d-aac4-93df3474505e service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:55.152861 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a076cfd-8347-4c8d-aac4-93df3474505e service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:55.153131 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a076cfd-8347-4c8d-aac4-93df3474505e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:55.154855 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0a076cfd-8347-4c8d-aac4-93df3474505e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:55.166841 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a076cfd-8347-4c8d-aac4-93df3474505e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:55.168458 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a076cfd-8347-4c8d-aac4-93df3474505e service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:55.178070 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1476/2921] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:14:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:15:05.173661 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=6 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:58.303215 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b15c8e-dee8-45a9-9bd9-1dd39871d943 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:58.304724 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-51b15c8e-dee8-45a9-9bd9-1dd39871d943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:58.316797 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b15c8e-dee8-45a9-9bd9-1dd39871d943 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:58.320918 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33130005-4ed4-4af5-b5c5-449ba2739598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:58.322227 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33130005-4ed4-4af5-b5c5-449ba2739598 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:58.322227 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33130005-4ed4-4af5-b5c5-449ba2739598 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:58.322227 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33130005-4ed4-4af5-b5c5-449ba2739598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:58.323910 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-33130005-4ed4-4af5-b5c5-449ba2739598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:58.334295 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33130005-4ed4-4af5-b5c5-449ba2739598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:58.335754 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33130005-4ed4-4af5-b5c5-449ba2739598 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:58.344152 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1446/2922] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:16:08.340307 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=6 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:16:28.085151 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a076cfd-8347-4c8d-aac4-93df3474505e service nova] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:16:28.135854 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a076cfd-8347-4c8d-aac4-93df3474505e service nova] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:16:28.141016 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0c31f-3656-4c2a-a500-02e143105ee1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:16:28.142022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0c31f-3656-4c2a-a500-02e143105ee1 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:28.142022 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0c31f-3656-4c2a-a500-02e143105ee1 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:28.142257 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0c31f-3656-4c2a-a500-02e143105ee1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:28.143960 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-37b0c31f-3656-4c2a-a500-02e143105ee1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:16:28.179455 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0c31f-3656-4c2a-a500-02e143105ee1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:16:28.181284 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0c31f-3656-4c2a-a500-02e143105ee1 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:16:28.192276 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1477/2923] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:16:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:16:28.550682 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33130005-4ed4-4af5-b5c5-449ba2739598 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:16:28.590061 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33130005-4ed4-4af5-b5c5-449ba2739598 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:16:28.593806 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b315a8d-122d-4bb9-b696-a3e7f1497464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:16:28.594621 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b315a8d-122d-4bb9-b696-a3e7f1497464 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:28.594919 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b315a8d-122d-4bb9-b696-a3e7f1497464 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:28.595815 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b315a8d-122d-4bb9-b696-a3e7f1497464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:28.596971 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0b315a8d-122d-4bb9-b696-a3e7f1497464 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:16:28.611889 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b315a8d-122d-4bb9-b696-a3e7f1497464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:16:28.613328 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b315a8d-122d-4bb9-b696-a3e7f1497464 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:16:28.625275 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1447/2924] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:16:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:16:29.811196 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0c31f-3656-4c2a-a500-02e143105ee1 service nova] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:16:29.821729 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0c31f-3656-4c2a-a500-02e143105ee1 service nova] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:16:29.825416 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a09c7-3342-4d49-ac50-6852cb0d03c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:16:29.826489 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a09c7-3342-4d49-ac50-6852cb0d03c9 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:29.826489 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a09c7-3342-4d49-ac50-6852cb0d03c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:29.826732 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a09c7-3342-4d49-ac50-6852cb0d03c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:29.828293 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7e2a09c7-3342-4d49-ac50-6852cb0d03c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:16:29.842852 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a09c7-3342-4d49-ac50-6852cb0d03c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:16:29.855829 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a09c7-3342-4d49-ac50-6852cb0d03c9 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:16:29.856192 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1478/2925] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:16:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:16:30.745398 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b315a8d-122d-4bb9-b696-a3e7f1497464 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:16:30.757231 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b315a8d-122d-4bb9-b696-a3e7f1497464 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:16:30.763070 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56e08a-b909-4d2b-b0c0-e4543aa20266 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38127f41852d412ea6ad270f1618fce2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:16:30.763662 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56e08a-b909-4d2b-b0c0-e4543aa20266 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:30.763940 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56e08a-b909-4d2b-b0c0-e4543aa20266 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:30.764212 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56e08a-b909-4d2b-b0c0-e4543aa20266 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:30.765944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8d56e08a-b909-4d2b-b0c0-e4543aa20266 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:16:30.787376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d56e08a-b909-4d2b-b0c0-e4543aa20266 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:16:30.787376 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d56e08a-b909-4d2b-b0c0-e4543aa20266 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:16:30.790862 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1448/2926] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:16:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:16:39.851277 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=34 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:16:40.786360 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=24 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:16:44.263921 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a09c7-3342-4d49-ac50-6852cb0d03c9 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:16:44.272676 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a09c7-3342-4d49-ac50-6852cb0d03c9 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:16:44.276299 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af43e37-c3fe-4f0c-9331-bbe98f849f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdbbaa64ad4844a0b2698dee82f95f23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:16:44.276893 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af43e37-c3fe-4f0c-9331-bbe98f849f80 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:44.277167 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af43e37-c3fe-4f0c-9331-bbe98f849f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:44.277436 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af43e37-c3fe-4f0c-9331-bbe98f849f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:44.279169 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0af43e37-c3fe-4f0c-9331-bbe98f849f80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:16:44.363545 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af43e37-c3fe-4f0c-9331-bbe98f849f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:16:44.365033 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af43e37-c3fe-4f0c-9331-bbe98f849f80 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:16:44.384339 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1479/2927] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:16:47.680501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56e08a-b909-4d2b-b0c0-e4543aa20266 service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:16:47.681944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8d56e08a-b909-4d2b-b0c0-e4543aa20266 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:16:47.696536 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56e08a-b909-4d2b-b0c0-e4543aa20266 service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:16:47.701639 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a74fba-2419-43bb-a7fd-50353ecd3502 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9506139d5744fe69153e7e30a05b436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:16:47.702181 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a74fba-2419-43bb-a7fd-50353ecd3502 service placement] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:47.702465 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a74fba-2419-43bb-a7fd-50353ecd3502 service placement] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:47.702743 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a74fba-2419-43bb-a7fd-50353ecd3502 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:47.704501 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-79a74fba-2419-43bb-a7fd-50353ecd3502 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:16:47.716650 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a74fba-2419-43bb-a7fd-50353ecd3502 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:16:47.718252 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a74fba-2419-43bb-a7fd-50353ecd3502 service placement] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:16:47.729185 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1449/2928] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:16:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:16:51.341096 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af43e37-c3fe-4f0c-9331-bbe98f849f80 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:16:51.351658 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af43e37-c3fe-4f0c-9331-bbe98f849f80 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:16:51.357253 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f5b6e-b510-4668-a32d-fc2eb226715c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9506139d5744fe69153e7e30a05b436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:16:51.357860 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f5b6e-b510-4668-a32d-fc2eb226715c service placement] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:51.358139 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f5b6e-b510-4668-a32d-fc2eb226715c service placement] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:51.358420 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f5b6e-b510-4668-a32d-fc2eb226715c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:51.360962 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ba4f5b6e-b510-4668-a32d-fc2eb226715c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:16:51.373626 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f5b6e-b510-4668-a32d-fc2eb226715c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:16:51.375814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f5b6e-b510-4668-a32d-fc2eb226715c service placement] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:16:51.386350 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1480/2929] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:16:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:16:51.464336 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a74fba-2419-43bb-a7fd-50353ecd3502 service placement] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:16:51.466289 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-79a74fba-2419-43bb-a7fd-50353ecd3502 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:16:51.482369 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a74fba-2419-43bb-a7fd-50353ecd3502 service placement] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:16:51.489210 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25473178-b5c1-47a9-b367-0915b463ce19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:16:51.489837 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25473178-b5c1-47a9-b367-0915b463ce19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:51.490111 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25473178-b5c1-47a9-b367-0915b463ce19 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:51.490379 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25473178-b5c1-47a9-b367-0915b463ce19 admin admin] PATH_INFO: `/v3/users/4b8776b82114444cbe35aa534fe14fa2` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:51.492024 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25473178-b5c1-47a9-b367-0915b463ce19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b8776b82114444cbe35aa534fe14fa2)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:16:51.493913 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25473178-b5c1-47a9-b367-0915b463ce19 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:16:51.670031 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-25473178-b5c1-47a9-b367-0915b463ce19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b8776b82114444cbe35aa534fe14fa2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:51.724450 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-25473178-b5c1-47a9-b367-0915b463ce19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b8776b82114444cbe35aa534fe14fa2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:51.741908 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-25473178-b5c1-47a9-b367-0915b463ce19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b8776b82114444cbe35aa534fe14fa2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:51.742651 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-25473178-b5c1-47a9-b367-0915b463ce19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b8776b82114444cbe35aa534fe14fa2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:51.749988 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-25473178-b5c1-47a9-b367-0915b463ce19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b8776b82114444cbe35aa534fe14fa2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:51.756115 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1450/2930] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:16:51 2022] DELETE /identity/v3/users/4b8776b82114444cbe35aa534fe14fa2 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:16:52.034859 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f5b6e-b510-4668-a32d-fc2eb226715c service placement] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:16:52.050554 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f5b6e-b510-4668-a32d-fc2eb226715c service placement] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:16:52.111641 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c083ef8-6be5-4247-87bc-8b44660def92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:16:52.112266 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c083ef8-6be5-4247-87bc-8b44660def92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:52.112544 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c083ef8-6be5-4247-87bc-8b44660def92 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:52.112814 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c083ef8-6be5-4247-87bc-8b44660def92 admin admin] PATH_INFO: `/v3/projects/f7224a31a8aa4e7ca9514d77b5f00526` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:52.113979 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c083ef8-6be5-4247-87bc-8b44660def92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7224a31a8aa4e7ca9514d77b5f00526)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:16:52.115921 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c083ef8-6be5-4247-87bc-8b44660def92 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:16:52.245073 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8c083ef8-6be5-4247-87bc-8b44660def92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7224a31a8aa4e7ca9514d77b5f00526', 'request_id': 'req-8c083ef8-6be5-4247-87bc-8b44660def92'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:52.248780 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:16:52.323701 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8c083ef8-6be5-4247-87bc-8b44660def92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7224a31a8aa4e7ca9514d77b5f00526', 'request_id': 'req-8c083ef8-6be5-4247-87bc-8b44660def92'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:52.330433 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8c083ef8-6be5-4247-87bc-8b44660def92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7224a31a8aa4e7ca9514d77b5f00526', 'request_id': 'req-8c083ef8-6be5-4247-87bc-8b44660def92'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:52.342409 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8c083ef8-6be5-4247-87bc-8b44660def92 admin admin] The token cache is being invalidate because project f7224a31a8aa4e7ca9514d77b5f00526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:16:52.343035 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8c083ef8-6be5-4247-87bc-8b44660def92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:52.343370 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8c083ef8-6be5-4247-87bc-8b44660def92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:52.344844 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1481/2931] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:16:52 2022] DELETE /identity/v3/projects/f7224a31a8aa4e7ca9514d77b5f00526 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:16:52.349015 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25473178-b5c1-47a9-b367-0915b463ce19 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:16:52.350460 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-25473178-b5c1-47a9-b367-0915b463ce19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:16:52.364095 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25473178-b5c1-47a9-b367-0915b463ce19 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:16:52.423264 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd886e12-0489-4d07-9b16-dbc8363c4183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:16:52.423891 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd886e12-0489-4d07-9b16-dbc8363c4183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:52.424171 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd886e12-0489-4d07-9b16-dbc8363c4183 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:52.424440 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd886e12-0489-4d07-9b16-dbc8363c4183 admin admin] PATH_INFO: `/v3/users/336431a86fcc46c4b62f225d5abef579` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:52.425553 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd886e12-0489-4d07-9b16-dbc8363c4183 admin admin] RBAC: Authorizing `identity:delete_user(user_id=336431a86fcc46c4b62f225d5abef579)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:16:52.427526 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd886e12-0489-4d07-9b16-dbc8363c4183 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:16:52.582479 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd886e12-0489-4d07-9b16-dbc8363c4183 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '336431a86fcc46c4b62f225d5abef579'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:52.655090 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd886e12-0489-4d07-9b16-dbc8363c4183 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '336431a86fcc46c4b62f225d5abef579'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:52.660921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:16:52.669318 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd886e12-0489-4d07-9b16-dbc8363c4183 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '336431a86fcc46c4b62f225d5abef579'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:52.669978 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd886e12-0489-4d07-9b16-dbc8363c4183 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '336431a86fcc46c4b62f225d5abef579'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:52.677549 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd886e12-0489-4d07-9b16-dbc8363c4183 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '336431a86fcc46c4b62f225d5abef579'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:52.683594 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1451/2932] 10.222.0.91 () {58 vars in 1224 bytes} [Thu Sep 8 14:16:52 2022] DELETE /identity/v3/users/336431a86fcc46c4b62f225d5abef579 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:16:52.914473 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c083ef8-6be5-4247-87bc-8b44660def92 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:16:52.916273 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8c083ef8-6be5-4247-87bc-8b44660def92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:16:52.930924 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c083ef8-6be5-4247-87bc-8b44660def92 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:16:52.992560 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4221b-51c2-42cb-bd2e-0c1da959ea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c14eb0bf645e49f887a8d288d37d1491', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '189571d729d54bff900838adb0030c5b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:16:52.993214 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4221b-51c2-42cb-bd2e-0c1da959ea59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:52.993493 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4221b-51c2-42cb-bd2e-0c1da959ea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:52.993767 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4221b-51c2-42cb-bd2e-0c1da959ea59 admin admin] PATH_INFO: `/v3/projects/deac2c5d3b114aa88eb0426fbe824e05` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:52.994882 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4221b-51c2-42cb-bd2e-0c1da959ea59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deac2c5d3b114aa88eb0426fbe824e05)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:16:52.996864 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4221b-51c2-42cb-bd2e-0c1da959ea59 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:16:53.101136 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e3e4221b-51c2-42cb-bd2e-0c1da959ea59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deac2c5d3b114aa88eb0426fbe824e05', 'request_id': 'req-e3e4221b-51c2-42cb-bd2e-0c1da959ea59'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:53.169863 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e3e4221b-51c2-42cb-bd2e-0c1da959ea59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deac2c5d3b114aa88eb0426fbe824e05', 'request_id': 'req-e3e4221b-51c2-42cb-bd2e-0c1da959ea59'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:53.176604 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e3e4221b-51c2-42cb-bd2e-0c1da959ea59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deac2c5d3b114aa88eb0426fbe824e05', 'request_id': 'req-e3e4221b-51c2-42cb-bd2e-0c1da959ea59'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:53.188100 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e3e4221b-51c2-42cb-bd2e-0c1da959ea59 admin admin] The token cache is being invalidate because project deac2c5d3b114aa88eb0426fbe824e05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:16:53.188385 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e3e4221b-51c2-42cb-bd2e-0c1da959ea59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:53.193773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e3e4221b-51c2-42cb-bd2e-0c1da959ea59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:53.193773 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1482/2933] 10.222.0.91 () {58 vars in 1233 bytes} [Thu Sep 8 14:16:52 2022] DELETE /identity/v3/projects/deac2c5d3b114aa88eb0426fbe824e05 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:17:02.803980 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=9,keystone:DELETE=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:17:03.186484 dv-cinder-847733-4-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:19:10.670675 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd886e12-0489-4d07-9b16-dbc8363c4183 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:19:10.672596 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fd886e12-0489-4d07-9b16-dbc8363c4183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:19:10.782593 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd886e12-0489-4d07-9b16-dbc8363c4183 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:19:10.846726 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f77c2-eb42-44ac-963f-494797e4bc1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9506139d5744fe69153e7e30a05b436', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3fb2bd53d40841eeb1653bd5927fb189', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:19:10.847374 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f77c2-eb42-44ac-963f-494797e4bc1a service placement] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:19:10.847648 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f77c2-eb42-44ac-963f-494797e4bc1a service placement] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:19:10.847921 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f77c2-eb42-44ac-963f-494797e4bc1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:19:10.849725 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-979f77c2-eb42-44ac-963f-494797e4bc1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:19:10.861683 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f77c2-eb42-44ac-963f-494797e4bc1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:19:10.863348 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979f77c2-eb42-44ac-963f-494797e4bc1a service placement] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:19:10.875100 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1452/2934] 10.222.0.91 () {60 vars in 1365 bytes} [Thu Sep 8 14:19:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:19:20.869944 dv-cinder-847733-4-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=67 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}